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FOREWORD 

This  is  Part  V  of  a  series  of  reports  on  rationales  and 
techniques  of  matrix  factoring  which  play  an  important  role 
in  multivariate  analysis  techniques.  Indeed,  it  may  well  be 
said  that  all  adequate  models  and  methods  of  multivariate 
analysis  are  special  cases  of  matrix  factoring  techniques. 

The  more  traditional  methods  of  factor  analysis,  in  particular, 
are  special  cases  of  more  general  matrix  factoring  techniques, 
as  are  also  all  multiple  regression  models. 
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CHAPTER  17 


PRIMARY  FACTOR  MATRICES  FROM  HYPOTHESES 

We  have  seen  In  the  previous  chapters  that  we  may  calculate,  in  a  wide 
variety  of  ways,  a  factor  loading  matrix  whose  major  product  moment  gives  a 
lower  rank  approximation  to  the  correlation  or  covariance  matrix  obtained 
from  data  matrices.  We  also  know  from  Chapter  4  that,  if  any  of  these  matri¬ 
ces  obtained  by  a  particular  computational  method  were  postmultiplied  by  a 
square  orthonormal  matrix,  the  major  product  moment  of  the  resulting  matrix 
would  be  the  same  as  the  major  product  moment  of  the  matrix  prior  to  multi¬ 
plication  by  the  orthonormal  matrix.  This,  of  course,  is  because  in  the 
major  product  moment  the  square  orthonormal  matrix  is  multiplied  by  its 
transpose  to  yield  the  identity  matrix. 

We  also  saw  in  Chapter  4  that,  if  we  regard  the  factor  loading  matrix 
as  one  of  the  factors  in  the  product  of  the  two  matrices  which  purports  to 
approximate  the  data  matrix,  the  same  situation  prevails.  Suppose  we  have 
some  approximation  to  a  factor  score  matrix,  postmultiplied  by  the  trans¬ 
pose  of  a  factor  leading  matrix,  as  a  lower  rank  approximation  to  the  data 
matrix.  We  may  then  also  have  the  factor  score  matrix  postmultiplied  by  an 
orthonormal  matrix  to  yield  another  factor  score  matrix,  and  the  transpose 
of  the  factor  loading  matrix  premultiplied  by  the  transpose  of  the  same  ortho 
normal  matrix.  Then  the  product  of  these  two  transformed  matrices  would  be 
exactly  the  same  as  that  of  the  original  matrices. 

Furthermore ,  we  learned  that  these  transformation  matrices  need  not  be 
square  orthonormal,  but  that  we  may  have  a  more  general  situation.  We  may 
have  the  factor  score  matrix  postmultiplied  by  some  square  basic  matrix,  and 


the  transpose  of  the  factor  loading  matrix  premultiplied  by  the  inverse  of 
this  same  matrix.  Then  the  major  product  of  the  two  transformed  matrices 
would  be  the  same  as  the  majqp  product  of  the  two  original  matrices.  This 
is  because  the  square  basic  matrix  multiplied  by  its  inverse  yields  the 
identity  matrix. 

It  is  clear  then,  that  we  may  have  a  multiply  infinite  number  of  factor 
loading  matrices  whose  major  product  moments  give  identical  results.  Simi¬ 
larly,  we  may  have  an  infinite  number  of  pairs  of  factor  score  and  factor 
loading  matrices,  whose  major  products  give  identical  results.  The  question 
then  arises  as  to  which  of  these  pairs  of  factor  loading  and  factor  score 
matrices  is  best,  in  some  defined  sense. 

In  this  chapter  we  shall  consider  only  factor  loading  matrices.  We 
shall  attempt  to  achieve  seme  transformation  of  the  arbitrary  matrix  so  that 
the  new  factor  loading,  matrix  will  have  the  following  characteristics .  First 
for  each  factor  loading  vector,  only  a  relatively  small  number  of  the  vari¬ 
ables  shall  have  high  loadings,  and  the  remainder  should  have  small  load¬ 
ings.  Second,  each  variable  shall  have  loadings  in  only  a  few  of  the  factors 
Third,  for  any  given  pair  of  factors,  a  number  of  the  variables  shall  have 
small  loadings  in  both  factors.  Fourth,  for  any  given  pair  of  factors,  seme 
of  the  variables  shall  have  high  loadings  in  one  factor  and  low  in  the  other, 
while  other  variables  shall  have  high  loadings  in  the  second  factor  but  not 
in  the  first.  Fifth,  for  any  given  pair  of  factors,  very  few  of  the  vari¬ 
ables  shall  have  high  loadings  in  both. 

These  are  the  conditions  which  Thurstone  (19^7)  has  formulated  as  the 
" simple  structure"  criteria.  We  shall  therefore  refer  to  transformations 


which  attempt  to  achieve  these  objectives  as  simple  structure  transforma¬ 
tions,  and  we  shall  refer  to  the  transformed  factor  loading  matrix  as  a 
simple  structure  factor  loading-,  matrix . 

There  are-,  i-n  general-,-  three  kinds  of  methods  which  have  been  used  for 
transforming  arbitrary  factor  matrices  to  simple  structure  matrices.  The 
first  of  these  is  by  means  of  graphical  methods  of  rotation.  In  this  method, 
every  vector  of  factor  loadings  is  plotted  against  every  other  vector,  and 
by  inspection  of  the  plots,  rotations  or  transformations  are  made  two  at  a 
time.  This  is  the  oldest  of  the  methods.  It  was  developed  by  Thurstone 
(19^7)  and  has  been  used  extensively.  The  chief  disadvantages  of  the  method 
are  that  it  is  extremely  time-consuming;  it  is  not  adapted  to  objective  com¬ 
putational  routines;  and  a  great  deal  of  personal  judgment  is  left  to  the  in- 
dividual  who  does  the  plotting  and  the  transformations . 

The  second  procedure  is  based  on  a  priori  hypotheses.  Here  the  invest! » 
gator  has  sane  a  priori  hypothesis  as  to  which  variables  should  have  high 
loadings  in  which  factors,  and  which  should  not.  This  we  may  call  the  hypo¬ 
thesis  method  of  transformation . 

The  third  type  of  method  involves  analytical  or  mathematical  criteria 
for  transformation .  These  methods  adopt  certain  mathematical  functions  of 
the  transformed  factor  loadings  which  are  to  be  optimized.  They  are  called 
the  analytical  methods  of  transformation. 

While  the  graphical  methods  have  been  extensively  used  in  the  past, 
they  have  been  falling  more  and  more  into  disuse  as  more  objective  methods 
have  been  developed.  Therefore,  these  methods  will  not  be  discussed  in  this 
book.  In  this  chapter,  we  shall  give  consideration  to  those  methods  which 


start  with  some  hypothesis  as  to  which  factor  loading  should  he  sizable  in 
which  variables.  In  the  next  chapter,  we  shall  consider  some  analytical 
methods  which  do  not  depend  on  a  priori  hypotheses. 

In  any  case,  it  must  be  emphasized,  as  Thurstone  (194?)  has  done  so 
frequently,  that  the  extent  to  which  simple  structure  can  be  achieved  by  any 
of  the  methods  is  definitely  limited  by  the  nature  of  the  data  to  be  analyzed 
and  that  simple  structure  must  be  inherent  in  the  data  if  any  of  the  methods 
is  to  reveal  it  in  the  transformed  factor  matrices. 

17.1  Characteristics  of  the  Hypothesis  Methods 

The  hypothesis  methods  of  transformation  are  similar  in  se/eral  respects 
First,  they  begin  with  some  arbitrary  factor  loading  matrix— that  is,  with 
some  factor  loading  matrix  computed  by  one  of  the  various  methods  outlined 
in  the  previous  chapters,  or  by  other  methods.  Second,  they  are  based  on 
some  hypothesis  as  to  which  of  the  tests  have  high  loadings  in  which  factors. 
The  methods  differ  essentially  in  the  types  of  transformation  matrices  em¬ 
ployed. 

17.1.1  The  Arbitrary  Factor  Matrices.  The  methods  all  begin  with  some 
arbitrary  factor  matrix  such  as  the  centroid,  the  multiple  group,  the  group 
centroid,  or  the  principal  axis  solution.  In  the  methods  we  have  discussed, 
it  will  be  recalled  that  they  are  all  special  cases  of  the  rank  reduction 
method.  It  is,  however,  not  necessary  that  the  arbitrary  factor  matrices 
be  special  cases  of  the  rank  reduction  method.  They  may  be  based  on  some 
clustering  or  B-coefficient  methods,  such  as  described  by  Holzinger  and 
Harman  (l94i). 

The  computational  routines  of  the  hypothesis  methods  differ  essentially 


5 


* 


..  according  to  which  particular  type  of  arbitrary  matrix  is  used.  It  should 

be  emphasized  at  this  point  that  most  of  the  procedures  we  have  outlined 
have  had  as  their  primary  objective  the  finding  of  a  factor  loading  matrix 
which,  wi-th  a  minimum  number  of  factors,  will  give  the  best  approximation 
to  the  correlation  or  covariance  matrix.  A  major  concern  has  been  to  find 
the  smallest  number  ef  factors  which,  with  a  satisfactory  degree  of  accuracy, 
can  reproduce  the  correlation  or  covariance  matrix  and,  indirectly,  the  data 
matrix.  This  objective  recognizes  the  finding  of  a  lower  rank  best  approxi - 
matt on  to  a  data  matrix  as  fundamental  to  all  scientific  investigations . 

17.1.2  The  Hypothesis  Matrix.  The  group  of  methods  considered  in  this 
chapter  depend  on  having  sane  a  priori  hypotheses  as  to  which  tests  should 
have  high  loadings  in  which  factors,  and  which  tests  should  have  low  load- 
ings.  These  methods  are  characterized  by  the  specification  of  a  binary 
*  hypothesis  matrix.  This  binary  matrix  has  a  1  in  the  ijth  position  if  the 

ith  test  has  a  higi  loading  in  the  Jth  factor.  Otherwise,  it  has  a  0  in 
this  position. 

This  binary  hypothesis  matrix  can  be  made  up  after  the  variables  are 
assembled.  However,  Thur stone  (1 9^7)t  Guttman  (1952),  and  other  have  em¬ 
phasized  that  it  is  better  first  to  make  up  an  hypothesis  matrix  and  then 
to  attempt  to  specify  variables  which  will  satisfy  the  hypothesis. 

The  ideal  binary  hypothesis  matrix  would  be  one  in  which  there  is  only 
a  single  1  in  each  row,  and  roughly  an  equal  number  of  1's  in  each  column. 
This  would  be  a  nonoverlapping  hypothesis  matrix.  However,  there  is  noth- 
ing  to  prevent  one  frcm  having  a  more  complex  hypothesis  so  that  he  may 
have  several  l's  in  each  row  for  some  of  the  variables,  indicating  that  he 


4 


believes  the  variables  have  loadings  on  more  than  one  of  the  factors. 

It  will  be  recalled  that  a  binary  matrix  is  also  involved  in  the  multiple 
group  method  of  factor  analysis,  and  that  binary  vectors  are  involved  in  the 
group  centroid  method.  In  those  methods,  however,  it  was  considered  desir¬ 
able,  but  not  mandatory,  that  these  binary  matrices  or  vectors  represent 
plausible  hypotheses  as  to  the  factor  loadings  to  be  found. 

17.1.3  The  Transformation  Matrix.  The  methods  described  in  this  chapter 
all  require  a  decision  as  to  what  particular  conditions  are  to  he  satisfied 
by  the  transformation  matrix.  Usually  it  is  considered  desirable  to  have  a 
transformation  matrix  which  is  normal  by  columns.  That  is,  the  minor  product 
moment  of  the  transformation  matrix  should  have  unity  in  the  diagonals.  This 
restriction  is  required  so  that  the  factor  vectors  of  the  new  transformed 
factor  matrix  will  be  comparable  to  one  another.  More  technically,  such  a 
transformation  provides  the  basis  for  finding  the  correlation  among  the  new 
reference  axes  from  which  the  new  factor  loadings  are  measured. 

This  concept,  however,  involves  us  with  geometric  and  trigonometric 
symbolic  systems  which  we  wish  to  avoid  in  this  book.  We  have  attempted  to 
restrict  ourselves  to  algebraic  and  arithmetic  concepts.  This  has  been  in 
the  belief  that  the  traditional  random  mixture  of  various  types  of  symbolic 
mathematical  systems  does  not  yield  a  better  understanding  of  the  phenomena 
under  study,  unless  one  is  already  thoroughly  familiar  with  the  standard 
symbolic  systems  and  the  interrelationships  among  them.  The  assumption  in 
this  hook  is  that  many  readers  are  not  thoroughly  familiar  with  these  over¬ 
lapping  and  interrelated  symbolic  systems. 

The  methods  differ  with  respect  to  the  type  of  transformation  involved. 
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fhes$  may  be  square  orthonormal  transformations.  If  the  transformations 
are  not  orthonormal,  they  are  called  oblique.  Therefore,  a  choice  between 
orthonormal  and  oblique  transformations  is  available.  Mthoutjgh  the  term 
" oblique"  is  not  particularly  appropriate ,  we  shall  continue  to  use  it 
since  it  is  well-established  in  the  literature. 

The  methods  differ  also  in  restrictions  which  may  be  placed  on  the 
transformed  simple  structure  factor  loading  matrix.  Me  shall  describe  one 
method.  More  specifically,  one  type  of  transformation  is  such  that,  for  all 
variables  which  have  0's  in  a  given  column  of  the  hypothesis  binary  matrix, 
the  sum  of  their  loadings  in  the  simple  structure  matrix  for  the  correspond¬ 
ing  factor  vector  is  0. 

17.2  Kinds  of  Methods 

In  this  chapter  we  shall  outline  three  methods  which  differ  with  respect 
to  the  type  of  arbitrary  matrix  on  which  they  are  based.  A  fourth  method 
will  yield  a  zero  sum  for  assumed  zero  loadings  in  each  factor.  A  fifth 
method  uses  a  square  orthonormal  transformation. 

17.2.1  The  Multiple  Group  Matrix.  This  method  begins  with  an  arbitrary 
factor  matrix  obtained  by  the  multiple  group  method.  We  shall  assume  that 
the  binary  matrix  used  in  the  multiple  group  method  was  actually  an  hypo¬ 
thesis  matrix,  and  that  it  is  the  one  which  the  transformed  simple  structure 
matrix  is  to  resemble  as  closely  as  possible.  This  particular  type  of  arbi¬ 
trary  factor  loading  matrix  Is  regarded  as  a  special  case  because  certain 
computational  simplifications  are  possible  when  the  binary  grouping  matrix 
and  the  binary  hypothesis  matrix  are  the  Same. 

17.2.2  The  Principal  Axis  Arbitrary  Matrix.  The  second  method  we  shall 
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outline  uses  the  principal  axis  or  basic  structure  factor  loading  matrix 
as  the  arbitrary  matrix.  In  many  cases  when  high  speed  computers  have  been 
used,  the  solution  will  be  a  principal  axis  or  basic  structure  factor  matrix. 
®he  computations  for  the  transformed  Simple  structure  matrix  are  simplified 
because  the  principal  axis  solution  yields  an  orthogonal  factor  matrix. 

17.2.5  the  Arbitrary  Factor  Matrix,  this  method  may  be  regarded  as  a 
generalization  of  other  methods,  of  which  the  multiple  group  and  the  princi¬ 
pal  axis  arbitrary  matrices  are  special  cases.  However,  the  solution  does 
not  depend  on  any  peculiar  properties  of  the  factor  loading  matrix,  such  as 
orthogonality,  or  on  the  identity  of  the  hypothesis  and  the  grouping  binary 
matrix,  as  in  the  multiple  group  method.  The  method  does  not  depend  on  how 
the  arbitrary  factor  matrix  was  determined,  and  requires  only  the  construc¬ 
tion  of  the  binary  hypothesis  matrix. 

17.2.4  The  Zero  Partial  Sum  Method.  This  method  is  independent  of  the 
particular  type  of  arbitrary  matrix  on  which  it  is  based.  We  may  begin  with 
any  of  the  solutions  discussed  in  the  previous  chapters.  In  any  case,  the 
transformation  of  the  arbitrary  matrix  yields  a  simple  structure  matrix  such 
that  those  variables  which  have  zeros  in  an  hypothesis  vector  will  yield  a 
zero  sum  for  the  corresponding  factor  loadings.  This  does  not  mean  neces¬ 
sarily  that  the  simple  structure  criteria  may  be  well  satisfied,  or  that  the 
binary  hypothesis  matrix  may  be  reasonably  well  approximated  by  the  trans¬ 
formed  matrix.  Por  example,  it  is  quite  possible  that  even  though  partial 
sums  are  0,  the  elements  going  into  the  sum  may  still  vary  greatly.  Further¬ 
more,  appreciable  or  high  loadings  may  not  appear  in  the  simple  structure 
matrix  to  correspond  with  l's  in  the  hypothesis  matrix.  The  results  yielded 


by  this  or  any  simple  structure  solution  are  as  much  a  function  of  the  data 
themselves  as  of  the  particular  method  used. 

17.2.5  The  Orthonormal  Transformation.  In  the  methods  discussed  above, 
the  type  of  transformation- -ybether  orthonormal  or  oblique— was  not  mentioned-. 
It  will  be  recalled  that  only  if  the  transformation  of  a  factor  loading  matrix 
is  orthonormal  will  the  major  product  moment  of  the  original  and  the  trans¬ 
formed  factor  loading  matrix  be  the  same.  There  is  nothing  in  the  computa¬ 
tional  procedures  of  the  methods  just  discussed  to  guarantee  that  the  trans¬ 
formation  matrix  will  be  orthonormal.  In  general,  it  will  be  oblique .  It 
should  be  emphasized  here  that  this  is  not  a  serious  objection  because,  as 
we  have  indicated  in  Chapter  4,  the  main  objective  is  not  to  reproduce  the 
correlation  or  covariance  matrix  with  the  major  product  moment  of  a  factor 
loading  matrix,  but  rather  to  reproduce  the  original  or  rescaled  data  matrix 
as  closely  as  possible  by  the  major  product  of  a  factor  score  and  a  factor 
loading  matrix.  Therefore,  if  we  get  a  given  factor  loading  matrix  and  trans¬ 
form  it  with  some  nonorthonormal  or  oblique  matrix,  we  can  always  transform 
the  factor  score  matrix  corresponding  to  it  by  the  inverse  of  this  transfor¬ 
mation,  so  that  the  major  product  of  the  two  will  be  the  same  as  the  major 
product  of  the  original  factor  score  and  factor  loading  matrices. 

However,  there  has  been  considerable  insistence  among  seme  research 
workers  that  transformations  be  orthonprmal,  and  it  is  of  interest  to  con¬ 
sider  a  method  which  will  guarantee  an  orthonormal  solution  based  on  the 
binary  hypothesis  matrix.  The  chief  advantage  of  the  orthonormal  transfor¬ 
mation  procedures  is  that  in  certain  cases,  as,  for  example,  the  principal 
axis  method,  one  can  be  sure  that  the  factor  scores  are  uncorrelated,  or 
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what  amounts  to  the  same  thing,  that  the  factor  score  matrix  is  an  orthogonal 
matrix. 

IT  *3  Multiple  Group  Factor  -Matrix 

17.3.1  Characteristics  of  the  Method,  This  method  assumes  that  we  have 
adopted  the  multiple  group  method  of  factoring  the  correlation  matrix.  As 
will  be  recalled,  this  method  begins  with  the  multiplication  of  the  correla¬ 
tion  covariance  matrix  by  a  grouping  binary  matrix.  In  the  technique  of 
this  section,  the  grouping  binary  matrix  is  the  same  as  the  binary  hypothesis 
matrix. 

However,  the  procedure  is  such  that  one  does  not  need  to  carry  out  all 
of  the  computations  for  the  multiple  group  factor  matrix.  Because  the  group¬ 
ing  and  the  hypothesis  binary  matrices  are  the  same,  one  can  emit  seme  of  the 
computations  for  the  multiple  group  matrix.  Distinguishing  characteristics 
of  this  method  are  that  one  begins  with  only  a  partial  solution  of  the  multi¬ 
ple  group  matrix,  and  that  the  computations  are  somewhat  simpler  than  for 
other  hypothesis  methods  discussed  in  this  chapter. 

17.3.2  Computational  Equations 

17. 3* 2a  Definition  of  Notation 

R  is  the  correlation  matrix. 

f  is  the  binary  hypothesis  matrix. 

I  is  the  transformation  matrix. 

r  is  the  correlation  of  the  reference  axes. 

*■» 

b  is  the  simple  structure  factor  matrix. 
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17.3.2b  The  Equations 


F  =  8  f 
S  =  F'  f 

tt'  =  s 
G  =  .  F'  P 
C  =  G4  S 


(17.3.1) 

(17.3.2) 

(17.3.3) 

(17.3. 4) 

(17.3.5) 


g  =  s  c 

7  *  g  C 

D  - 

H  *=  t'  C  D 


r  =  Dr  D 

b  =  P  (C  D) 


(17.3.6) 

(17.3.7) 

(17.3.8) 

(17.3.9) 

(17.3.10) 

(17.5.11) 


17.3.3  Computational  Instructions.  We  begin  the  computations  with  the 
correlation  or  covariance  matrix,  rather  than  vith  the  multiple  group  factor 
matrix,  because  we  Shall  emit  the  final  steps  of  the  multiple  group  solution. 
We  assume  that  the  binary  matrix  f  has  been  constructed.  It  has  1’ s  for 
those  tests  in  each  factor  which  are  assumed  to  have  high  loadings,  and  0’s 
for  all  of  the  others. 

The  first  step,  as  in  the  multiple  group  method,  is  indicated  in  Eq. 
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(17.3.1) .  Here  we  postmultiply  a  correlation  matrix  by  the  hypothesis  binary 
matrix  f.  Obviously,  this  merely  serves  to  sum  those  columns  of  the  correla¬ 
tion  matrix  ..corresponding,  to  the  l’s  in  the  vectors  of  the  f  matrix  on  the 
ri$vt  of  Eq>  (if  .3,1^, 

The  next  step,  as  in  the  multiple  group  method,  is  to  premultiply  the  f 
matrix  by  the  transpose  of  the  F  matrix  calculated  in  Eq.  (17.3*1)*  This  is 
indicated  in  Eq.  (17.3.2)  by  the  S  matrix  on  the  left  of  the  equation.  This, 
in  effect,  adds  rows  of  the  F  matrix  from  Eq.  (17. 3.1),  corresponding  to  the 
l's  in  the  vectors  of  the  f  matrix,  to  give  a  symmetric  matrix,  S,  in  Eq. 

(17.3.2) . 

Eq.  (17.3.3)  indicates  a  triangular  factoring  of  the  matrix  S  of  Eq. 

(17.3.2) .  This  triangular  factoring  is  not  necessary  unless  one  actually 
wishes  to  see  the  transformation  matrix  which  is  indicated  in  Eq.  (17.3.9). 
For  most  of  the  parameters  of  interest  in  a  factor  analysis,  this  transfor- ■ 
mation  matrix,  as  such,  is  not  required.  It  is  only  to  make  the  analysis 
complete  that  it  might  he  Included. 

Eq.  (17.3.4)  is  the  minor  product  moment  of  the  F  matrix  calculated  in 
Eq.  (17.3.1).  This  is  indicated  on  the  left  by  G. 

Eq.  (17.3*5)  requires  the  calculation  of  the  inverse  of  the  G  matrix 
calculated  in  Eq.  (17,3.4).  The  matrix  designated  as  C  in  Eq.  (17.3.5)  is 
the  S  matrix  of  Eq.  (17,3.2)  premultiplied  by  the  inverse  of  the  G  matrix 
given  in  Eq.  (if *3,4). 

The  next  step  is  to  calculate  the  product  indicated  in  Eq.  (17.3.6). 
Here  we  praaultipiy  the  0  matrix  of  Eq.  (17.3.5)  by  the  S  matrix  of  Eq. 

(17.3.2) .  This  w  designate  as  g. 


We  next  require  the  7  matrix  indicated  in  Eq.  (17.3.7).  This  is  obtained 
by  premultiplying  the  C  matrix  of  Eq.  (17.3.5)  by  the  g  matrix  of  Eq.  (17.3.6). 

The  computation  of  a  diagonal  matrix  is  indicated  by  Eq.  (17.3.8).  This 
is  simply  a. matrix  whose  elements  are  the  reciprocal  square  roots  of  the  di¬ 
agonal  elements  of  the  7  matrix  given  by  Eq.  (17.3.7). 

The  computation  of  the  H  matrix,  which  is  the  transformation  matrix  for 
the  multiple  group  factor  matrix,  is  indicated  in  Eq.  (17,3.9).  This  is  the 
triple  product,  from  left  to  right,  of  the  upper  triangular  factor  of  the  S 
matrix  in  Eq.  (17.3*3)  by  the  C  matrix  of  Eq.  (17.3*5)  by  the  diagonal  matrix 
of  Eq,  (17,3.8).  As  indicated  above,  this  matrix  is  not  used  in  the  computa¬ 
tion  of  the  simple  structure  factor  loading  matrix.  It  is  used  in  Chapter  21 
for  the  calculation  of  general  factor  parameters,  but  if  these  are  not  de¬ 
sired  it  need  not  be  calculated. 

The  calculation  of  a  correlation  matrix  is  indicated  in  Eq.  ( 17.3*10). 
This  matrix  is  the  minor  product  moment  of  the  H  matrix  of  Eq.  (17.3*9).  It 
could  be  calculated  as  such  from  Eq.  (17.3.9),  but  it  is  simpler  to  calculate 
it  from  Eqs.  (17.3*7)  and  (17.3*8).  This  matrix  is  of  interest  because  it 
shows  the  extent  to  which  the  reference  vectors  of  the  transformation  matrix 
are  correlated.  It  is  precisely  a  correlation  matrix  of  the  simple  structure 
reference  axes.  In  most  factor  analyses,  both  this  matrix  and  its  normalized 
inverse  are  calculated  to  indicate  the  extent  to  which  the  transformation  de¬ 
parts  from  orthonormality.  More  particularly,  these  matrices  are  useful  for 
purposes  of  further  analysis,  as  indicated  in  Chapter  2,  The  matrix  is  of 
interest  also  because,  while  in  general  simple  structure  factor  matrices  on 
the  same  variables  are  supposed  to  be  relatively  invariant  from  one  type  of 


sample  to  another,  the  correlation  matrix  of  the  simple  structure  reference 
axes  as  given  in  Eq.  (17.3.IO)  may  he  regarded  as  characterizing  the  parti¬ 
cular  Semple  .on,  which  the  analysis  is  based.  It  may  vary  greatly  from  one 
sample  -to  another* 

The  calculation  of  the  simple  structure  factor  loading  matrix  is  given 
in  Eq.  (17.3.II),  As  indicated  on  the  right  of  this  equation,  we  compute 
first  a  matrix  which  is  the  product  of  the  C  matrix  of  Eq,  (17,5*5)  post- 
multlplied  by  the  diagonal  matrix  of  Eq.  (17.5.8).  This  matrix  is  then  post- 
muitiplied  into  the  F  matrix  of  Eq.  (17.5.1). 

The  characteristic  of  this  b  matrix  in  Eq.  (17.5.11)  is  that  it  should 
represent  as  nearly  as  possible  in  the  least  square  sense  the  simple  struc¬ 
ture  hypothesis  matrix  f  which  has  been  scaled  on  the  left  by  a  diagonal  such 
that  the  transformation  matrix  H  in  Eq.  (17.5*9)  is  normal  by  columns. 

17.5.4  Numerical  Example.  We  begin  this  numerical  example  with  the 
correlation  matrix  of  previous  chapters. 

We  shall  solve  for  only  three  factors;  therefore  the  binary  matrix  con¬ 
sists  of  only  three  column  vectors.  The  unit  elements  in  these  three  vectors 
are  taken,  respectively,  as  the  first  three,  the  second  three,  and  the  third 
three. 

Table  17.5.1  gives  the  correlation  matrix  postoultiplied  by  the  binary 
matrix  f,  as  indicated  by  Eq.  (17,3,1). 

Table  17.5.2  gives  the  minor  product  S  of  the  f  and  the  F  matrices,  as 
indicated  by  Eq.  (17.5.2). 

Table  17.5.3  gives  the  minor  product  moment  G  of  the  F  matrix,  as  in¬ 
dicated  by  Eq.  (17.3.4). 


Table  17, 3. k  gives  the  inverse  of  the  G  matrix  shown  in  Table  17.3.3* 

Table  17,3.5  gives  the  product  of  the  S  matrix  of  Table  17.3*2  pre- 
multiplied  by  the  G4  matrix  of  Table  17. 3. h.  This  product  is  the  C  matrix 
of  23c],  •  ('  17  «3 .5')  • 

In  the  body  of  Table  17.3,6  is  given  the  product  of  the  C  matrix  of 
Table  17.3*5;  preoultiplied  by  the  £  matrix  of  Table  17.3*2.  This  product 
is  the  g  matrix  of  Eq.  (17.3.6).  The  last  line  of  Table  17.3*6  is  obtained 
by  calculating  only  the  diagonal  elements  of  the  product  7  =  g  C  given  by 
Eq.  (17,3.7) ,  and  taking  the  reciprocal  square  roots  of  these  elements  as 
indicated  by  Eq.  (17,3,8). 

Table  17.3 «7  is  the  simple  structure  factor  loading  matrix  b.  This  is 
the  triple  product  of  the  matrix  F  given  by  Table  17*3.1;  the  matrix  C  given 
by  Table  17.3.5,  and  a  diagonal  matrix  D  constituted  from  the  elements  of 
the  last  row  of  Table  17.3*6.  This  product  is  indicated  by  Eq,  (17.3*11). 

The  computations  indicated  by  Eqs,  (17.3*3),  (17*3*9),  and  (17*3*1°) 
are  not  given  in  this  numerical  example. 
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Table  17.3.1  -  Matrix  E  f  =  F 


2,59700 

2.6o4O0 

oilS 

O.2990Q 

0.47100 

0.91700 

0.92700 

1.10500 


0.24900 

0.30100 

0.71500 

mmm 

2.33500 

0.57700 

0.37700 

0.91400 


0.95800 

1.03900 

Oifjaoo 

0v'80ioo 


2.19500 

2.06800 


Table  17.3.2  -  Matrix  f  f  =  S 


7.74400 

1.26500 

2.94900 


Table  17*3.3 
23.469158 

8.05789 

14.75028 


Table  17.3.4 
0.08804 

-o. 00167 

-0.07138 


Table  17.3.5 

0.46919 

-0.03173 

-0.204-21 


Table  17.3.6 


1.26500  2.94900 
6.94200  1.86800 
1.86800  6.44400 


-  Matrix  F'  F  =  0 

8.0-5789  14.75028 
18.04217  9.51620 

9.51620  17.97094 


-  Matrix  G1 

-0.00167  -0.07138 
0.07694  -0.03937 
-0.03937  0.13508 


-  Matrix  O*1  $  =  C 

-0.03358  -0.20345 
0.45844  -0.11490 
-0.11126  0.58641 


-  Matrix  S  C  =  g  and  Vector  of 

li  » 


-0.00818  0.00847 
2.93220  0.04042 
0.04042  2,96422 


0.864  O.76O 


i6«a 


Table  17.3.7  -  Simple  Structure  Factor  Loading  Matrix 
b  =  F  (C  D) 

0.8,57  -©'.©69  0.004 

0.84-5  -©*©56  ©..-©34 

0.824  0.-118  -0.0.31 

-0.003  0.804  0.083 

-0.028  0.874  -0.036 

0.0^  0.835  -0.016 

•  -0.028  -0.008  0.780 

-o.om  -0.089  0.802 
0.057  0.131  0.671 


IT 

» 

»  17*4  The  Principal  Axis  Factor  Matrix 

17.4.1  Characteristics  of  the  Method.  This  method,  like  the  previous 
one,  is  a  special  case  of  an  arbitrary  matrix  resulting  in  simplified  compu¬ 
tations.  Here  the  simplification  results  because  the  principal  axis  factor 
matrix  is  orthogonal  by  columns. 

One  of  the  most  important  characteristics  of  the  method  is  that  the 
computations  do  not  call  for  the  calculation  of  the  inverse  of  any  matrix 
except  a  diagonal  matrix.  This  is  far  easier  to  calculate  than  the  inverses 
of  symmetric  or  other  square  matrices.  The  method  is  of  considerable  practi - 
cal  importance  because,  with  the  increasing  availability  of  high  speed  com¬ 
puters,  most  of  the  arbitrary  factor  loading  matrices  calculated  will  be  of 
9  the  principal  axis  or  basic  structure  type. 

17.4.2  The  Computational  Equations 

*  17.4.2a  Definition  of  Notation 

a  is  the  principal  axis  factor  matrix. 

6  is  that  part  of  the  basic  diagonal  matrix  of  R  corresponding  to  a. 

r  is  the  correlation  matrix  of  the  reference  axes. 

b  is  the  simple  structure  factor  loading  matrix. 

17.4.2b  The  Equations 

G  =  a;  f  (17.4.1) 

C  =  6*1  G  (17.4.2) 

S 


*  c'  c 


(17.4.3) 


i 
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* 

D  -  J&0 

(17.4.4) 

H  *  CD 

(17.4.5) 

r  =  D  S  D 

(17*4.6) 

b  =  a  H 

(17.4.7) 

17.4.3  Computational  Instructions.  The  computations  in  this  method 
are  relatively  simple, 

We  begin  with  Eq.  (17*4.1).  This  is  simply  the  product  of  the  trans¬ 
posed  principal  axis  factor  loading  matrix  postmultiplied  by  the  binary  hypo¬ 
thesis  matrix  f. 

Next  we  calculate  the  C  matrix  in  Eq.  (17.4.2).  This  is  the  G  matrix 
of  Eq.  (17.4.1)  premultiplied  by  the  reciprocal  of  the  basic  diagonal  cor¬ 
responding  to  the  first  three  principal  axis  factors.  In  other  words,  this 
basic  diagonal  includes  only  the  basic  diagonal  elements  of  the  correlation 
matrix  corresponding  to  the  three  factors  which  have  been  solved  for. 

Next  we  calculate  Eq.  (17.4.3).  This  is  the  minor  product  moment  of 
the  C  matrix  calculated  in  Eq.  (17*4.2). 

We  then  calculate  a  diagonal  matrix  whose  elements  are  the  reciprocal 
square  roots  of  the  diagonal  elements  of  S  calculated  in  Eq.  (17.4.3).  This 
is  indicated  in  Eq.  (17.4.4). 

Next  we  calculate  the  transformation  matrix  H  as  in  Eq.  (17.4.5).  This 
is  the  matrix  G  of  Eq.  (17.4.2)  postmultiplied  by  the  diagonal  matrix  Of  Eq. 
(17.4.4).  It  will  be  noted  that  this  computation  was  optional  in  the  previous 
method*  Here,  however,  it  is  required  for  further  calculations,  as  will  be 


* 
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indicated  for  Eq.  (17.4,7). 

®he  next  step  is  to  calculate  the  correlation  of  the  simple  structure 
referencerascea,  :asrl<nai»c.ate,a-..iirn  Eq.  (17,4.6).  Here  the  matrix  a  of  Eq. 
(17.4,3)  is  pre-  end  p.ofit@.ul*tip.l£ied  hy  the  diagonal  matrix  of  Eq.  (17.4.4). 

As  seen  from  Eqs.  (17*4.3),  (17.4.4),  and  (17.4.5),  r  could  also  have  been 
calculated  by  taking  the  minor  product  moment  of  the  H  matrix  in  Eq.  (17.4,5). 
However,  this  would  have  meant  the  multiplication  of  square  matrices,  whereas 
the  computations  in  Eq.  (17.4.6)  require  only  operations  on  the  synmetric 
matrix  S  by  diagonal  matrices. 

Eq.  (17.4.7)  gives,  finally,  the  simple  structure  factor  loading  matrix. 
This  is  the  principal  axis  factor  loading  matrix  postmultiplied  by  the  trans¬ 
formation  matrix  H  of  Eq,  (17.4.5).  This  matrix  now  is  the  best  least  square 
approximation  to  the  scaled  binary  hypothesis  matrix  P,  in  which  the  scaling 
is  such  as  to  make  the  column  vectors  of  H  in  Eq.  (17.4.5)  normal. 

17.4.4  Numerical  Example.  We  take  as  numerical  data  for  this  illustra¬ 
tion  the  principal  axis  factor  loading  matrix  calculated  in  Chapter  8.  This 
is  repeated  for  convenience  in  Table  17.4.1,  together  with  the  first  three 
basic  diagonals  in  the  top  row.  The  binary  hypothesis  matrix  is  the  same  as 
given  in  the  previous  section.  This  hypothesis  assumes  that  no  variable  has 
factor  loadings  in  more  than  one  factor. 

Table  17.4.2  gives  the  product  of  the  transpose  of  the  matrix  in  Table 
17.4.1  postmultiplied  by  the  binary  hypothesis  matrix. 

Table  17.4.3  is  the  product  C  of  the  matrix  of  Table  17.4.2,  premulti - 
plied  hy  the  inverse  of  a  diagonal  matrix  constituted  from  the  basic  diagonal 
elements  at  the  top  of  Table  17.4.1.  These  calculations  are  indicated  by  Eq. 
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(17.4.2), 

Table  17.4.4  is  the  miner  product  moment  of  the  matrix  £  calculated  in 
Table  17.4.3* 

Table  17,4,5  is  the  matrix  obtained  by  postmulbi-plying  the  matrix  C  ©f 
Table  17.4.3  by  the  inverse  of  a  diagonal  matrix  whose  elements  are  from  the 
diagonal  of  the  matrix  in  Table  17.4.4.  This  gives  the  transformation  H  in¬ 
dicated  by  Eq.  (17.4.5). 

Table  17,4.6  is  the  simple  structure  factor  loading  matrix.  It  is  the 
product  of  the  principal  axis  factor  matrix  of  Table  17 .4.1,  postmultiplied 
by  the  matrix  H  of  Table  17.4.5.  This  product  is  indicated  by  Eq.  (17.4.7). 

The  correlation  matrix  r  of  Eq,  (17.4.6)  is  not  calculated  in  this 


numerical  example. 


J 
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fable  17.4.1  •  First  Three  Basic  Diagonals  and  Principal 
Axes  Factor  Vectors 

5*749 

2.050 

1*531 

0*717  0.493  0*350 

0.740  0.478  0.322 

0.775  0.29S  0.406 

0.556  -0.649  0,068 

0.463  -0.744  0.181 

0.518  -0.694  0.188 

0.640  0.080  -0.588  • 

0.615  O.166  -0.621 

0.715  -0.034  -0.369 


Table  17.4.2  -  Matrix  G  =  a1  f 

2.23000  1.53700  1.97000 
1.26700  -2.08700  0.21200 
1.07800  0.43700  -1.57800 


Table  17.4.3  -  Matrix  C  =  O’1  G 

0. 59483  0. 40998  0. 52547 
0.61805  -1.01805  0.10341 
0.80992  0.32832  -1.18557 


Table  17.4.4  -  Matrix  S  =  C;  C 

I.39177  -0.11-942  -0,58374 
-0.11942  1.31230  -0.27910 
-0.58374  -0.27910  1.69240 
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Table  17 ,4.5  -  Transformation  Matrix  H  =  C  D^fc 


0.40J92 

0.07949 

-0.91133 


Table  17,4.6  -  Simple  Structure  Matrix  b  =  a  H 

0.860  -0.081  0.010 

0.845  -0.068  0.043 

0.824  0.130  -0.034 

-0.013  0.795  0.111 

-0.032  0.879  -0.037 

0.027  0.856  -0.017 

-0.039  -0.011  0.801 

-0.029  -0.105  0.828 

0.089  0.180  0.622 


0.50420  O.3.5788 
0.523©  -0.88869 
0.68653  O.28661 


17*5  fiie  Arbitrary  Factor  Matrix 

17*5*1  Characteristics  of  the  Method.  This  method  is  more  general  than 
the  two  previous  methods  in  that  it  applies  to  any  factor  matrix,  including 
the  principal  axis,  centroid,  grouping,  and  multiple  group  matrices  as  special 
cases.  However,  the  method  includes  a  slight  modification  of  the  previous  two 
methods .  This  modification  is  desirable  in  most  transformation  solutions,  al¬ 
though  frequently  it  makes  very  little  difference, and  it  has  been  omitted  from 
the  two  previous  methods  in  order  to  simplify  computations. 

In  this  method,  the  arbitrary  factor  loading  matrix  is  normalized  by 
rows — that  is,  by  variables  —before  transformation  operations  begin  upon  it. 

The  reason  for  this  is  that  all  of  the  tests  will  then  be  given  equal  weight 
in  the  transformation  solution.  In  most  of  the  arbitrary  type  solutions  which 
are  designed  primarily  to  find  the  minimum  number  of  factors  which  can  ade¬ 
quately  account  for  a  correlation  or  data  matrix,  all  of  the  variables  do 
not  account  for  the  same  amount  of  variance  in  the  factor  matrix.  In  other 
words,  what  have  been  called  the  communal! ties  of  the  variables,  or  the  sums 
of  squares  of  factor  loadings  for  a  given  variable,  will  vary  considerably 
from  one  variable  to  another.  Therefore  it  is  considered  desirable  to  norm¬ 
alize  the  rows,  or  to  make  all  of  the  test  vectors  of  unit  length.  The  unit 
length  vector  will  be  recognized  by  some  as  a  geometric  concept.  We  shall 
not,  however,  develop  this  concept  further,  since  it  would  contribute  to  a 
confusion  of  symbolic  systems .  From  the  algebraic  or  arithmetic  point  of 
view  we  can  simply  state  that  we  wish  each  of  the  tests  to  carry  unit  weight 
in  the  determination  of  the  transformation.  It  may  be  recalled  that  in  Chapter 
15,  for  the  communal! ty  type  scaling  method,  the  a  factor  loading  matrices 
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satisfy  this  condition.  That  is,  for  any  given  number  of  factors,  the 
sums  of  squares  of  rows  of  the  a  matrix  are  all  equal  to  1. 

17.5.2  The  Computational  Equations 
17.5.2a  Definition  of  Notation 

a  is  an  arbitrary  factor  matrix. 


f  is  the  binary  hypothesis  matrix. 


r  is  the  correlation  of  the  simple  structure  reference  axes. 
H  is  the  simple  structure  transformation  matrix, 
b  i6  the  simple  structure  factor  matrix. 


17.5.2b  The  Equations 

D  -  dA' 

a  =  d  a 
G  =  A'  f 


C  =  A'  A 
M  =  C'1  G 


g  =  M;  M 


(17.5.1) 

(17.5.2) 

(17.5.3) 
(17.5.JO 

(17.5.5) 

(17.5.6) 


(17.5.7) 
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M  D 

(17.5.8) 

K'  H 

(17.5.9) 

A  H 

(17.5.10) 

17 . 5 • 3  Computational  Instructions.  In  this  method  we  assume  that  all 
of  the  computations  for  an  arbitrary  factor  matrix  have  been  completed  and 
that  we  start  with  this  arbitrary  matrix,  however  arrived  at. 

The  first  step  in  the  calculations  is  indicated  in  Eq.  (17.5.1).  Here 
we  calculate  a  diagonal  matrix  whose  elements  are  the  reciprocal  square  root 
of  the  diagonal  elements  of  the  major  product  moment  of  the  arbitrary  matrix 
a.  This  means,  of  course,  that  we  must  calculate  the  sums  of  squares  of  row 
elements  for  the  arbitrary  factor  matrix  a.  These  are  what  are  called  the 
commonalities  of  the  variables  based  on  the  particular  factoring  solution. 

The  second  step  is  to  calculate  the  normalized  factor  loading  matrix  A, 
as  indicated  in  Eq.  (17,5.2).  This  is  given  by  premultiplying  the  arbitrary 
factor  matrix  by  the  diagonal  matrix  in  Eq.  (17. 5*1)* 

Eq.  (17.5.3)  is  the  minor  product  of  the  matrix  in  Eq.  (17.5*2)  and  the 
hypothesis  matrix  f . 

We  then  calculate,  as  shown  in  Eq.  (17.5.U),  the  minor  product  moment 
of  the  matrix  A  given  by  Eq.  (17.5.2).  This  we  call  C. 

Next  we  calculate  a  matrix  M  which  is  the  G  matrix  of  Eq.  (17.5*3)  P*e- 
multiplied  by  the  inverse  of  the  matrix  0  in  Eq.  ( 3,7 . 5 *  ^ )  • 

We  then  calculate  Eq.  (17.5.6),  which  is  the  minor  product  moment  of  the 
matrix  calculated  in  Eq,  (17.5.5).  This  we  call  g. 


Next  we  calculate  the  diagonal  matrix  of  Eq.  (17.5.7).  This  is  simply 
a  matrix  whose  diagonal  elements  are  the  reciprocal  square  roots  of  the  di¬ 
agonal  elements  of  the  matrix  g  calculated  in  Eq.  (17,5.6). 

We  now  calculate  the  transformation  matrix  H  indicated  in  Eq.  (17.5.8). 
This  is  the  matrix  M  of  Eq.  (17.5*5)  postmultiplied  by  the  diagonal  matrix  of 
Eq.  (17.5.7).  The  method  of  computation  indicates  that  the  column  vectors 
of  the  H  matrix  are  normal. 

Eq.  (17.5.9)  gives  the  correlation  of  the  primary  reference  axes  as  the 
minor  product  moment  of  the  transformation  matrix  H  given  in  Eq.  (17.5.8). 

Finally,  Eq,  (17.5.IO)  gives  the  simple  structure  factor  loading  matrix. 
This  is  obtained  by  postmultiplying  the  matrix  of  Eq.  (17.5*2)  by  the  trans¬ 
formation  matrix  H  given  in  Eq.  (17.5.9). 

17.5*^  Numerical  Example 

Table  17.5.1  gives  the  centroid  factor  loading  matrix  calculated  in 
Chapter  6,  Section  5.  The  same  binary  hypothesis  matrix  is  used  as  in  the 
previous  two  methods. 

Table  17,5.2  gives  the  group  centroid  factor  matrix  normalized  by  rows, 
as  computed  by  Eqs.  (I7.5.I)  and  (17.5.2). 

Table  17.5*3  is  the  minor  product  G  of  the  normalized  factor  matrix  and 
the  binary  data  matrix,  as  computed  by  Eq.  (17.5*3) * 

Table  17.5,1).  is  the  minor  product  moment  C  of  the  normalized  factor 
matrix,  as  indicated  by  Eq.  (17.5.^). 

Table  17.5.5  is  the  inverse  c£  of  the  matrix  of  Table  17. 5 -k. 

Table  17.5.6  is  the  matrix  M  calculated  from  the  matrices  of  Tables 
17.5.3  and  17.5.5;  as  shown  in  Eq.  (17.5.5). 


Table  17.5*7  Is  the  transformation  matrix  H  obtained  by  normalizing 
the  columns  of  M  in  Table  17.5.6  by  means  of  Eqs.  (17. 5.7)  and  (17.5,8). 

Table  17.5*8  is  the  simple  structure  factor  matrix  b  obtained  from  the 
matrices  of  Tables  17.5.2  and  17,5.7,  indicated  by  Eq.  (17.5.10),  It  is  to 
be  noted  that  the  matrix  b  may  be  premultiplied  by  the  inverse  of  the  diagon 
al  matrix  of  Eq.  (17.5*1).  Shis  procedure  is  preferred  by  sane  factor 
analysts . 

The  correlation  matrix  r  of  Eq.  (17»5*9)  has  not  been  calculated. 
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fable  17.5.1  -  Group  Centroid  Factor  Matrix 


0.933 

-0.068 

0.002 

0.936 

-0.048 

0.033 

0.914 

0.115 

-0.035 

0.178 

0.840 

0.073 

0.107 

0.885 

-0.046 

0.169 

0.870 

-0.027 

O.33O 

O.I65 

0.777 

0.333 

0.087 

0.800 

0.397 

0.283 

0.667 

Table  I7.5.2  -  Normalized  Factor  Matrix  A 

-O.O7269  0.00214 
-O.O5II8  0.03519 
0.12475  -0.03797 
0.97476  0.08471 
0.99145  -0.05155 
0.98120  -0.03045 
0.19183  0.90333 
0.09990  Q. 91860 
0.34254  0.80732 


Table  17.5.5  -  Matrix  G  =  A'  f 

2.98688  0.51703  1.24654 
O.OOO87  2.94741  0.63426 
-0.00064  0.00273  2.62925 


Table  17.5.4  -  Matrix  C  «  a'  A 

3.59151  0.78369  1.09087 
0.78369  3.08345  0.53650 
1.09087  0.53650  2.32504 


Table  17.5.5  -  Matrix 

0.53511  -0.06023  -0.14353 
-0.06023  0.34870  -0.05220 
-0.14333  -0.05220  0.50939 


0.99735 

0.99807 

0.99146 

0.20656 

0.11987 

0.19060 

0.38366 

0.38237 

0.48052 
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Table  17,5.6  -  Matrix  M  =  C'1  G 


1.00098  -0.C0466 

0.00268 

-0.17957  0.99649 

O.OO883 

-0.42848  -0.-22658 

1.12755 

Table  17. 5.7  -  Transformation  Matrix  H  =  M 


0.90706  -0.00456  0.00257 
-0.16273  0.97510  0.00783 
-O.38828  -0.22171  0.99997 


Table  17.5.8  -  Simple  Structure  Factor  Matrix  b  = 


0.916  -0.076  0.004 
0,900  -0.062  0.037 
0.894  0.126  -0.035 
-0.004  0.931  0.093 
-0.033  0.978  -0.043 
0.025  0.963  -0.022 
-O.O34  -0.015  0.906 
-0.026  -O.IO8  0.920 
0.067  0.153  0.811 
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17,6  The  Zero  Partial  Sum  Transformation 

17.6.1  Characteristics  of  the  Method.  As  indicated  earli.er  in  this 
chapter,  the  rationale  of  this  procedure  is  to  find  a  transformation  such 
that,  for  any  given  factor  vector,  the  factor  loadings  in  the  simple  struc¬ 
ture  matrix  corresponding  to  the  zero  elements  in  the  corresponding  vector 
of  the  binary  hypothesis  matrix  shall  add  up  to  0.  This  restriction  can  he 
imposed  on  any  oblique  solution  for  any  arbitrary  factor  matrix.  It  applies 
equally  well  to  the  multiple  group,  principal  axis,  and  the  other  methods. 

In  general,  this  additional  restriction  tends  to  give  smaller  values 
for  the  near -zero  elements.  However,  there  might  still  be  considerable  dis¬ 
persion  about  the  mean  of  0  for  these  hypothesized  zero  elements.  In  any 
case,  the  computations,  as  will  be  seen,  are  somewhat  more  involved  than 
they  are  in  the  methods  previously  considered. 

17*6.2  The  Computational  Equations 
17.6.2a  Definition  of  Notation 


a  is  an  arbitrary  factor  matrix, 
f  is  the  binary  hypothesis  matrix. 

r  is  the  matrix  of  correlations  of  the  simple  structure  reference 
axes, 

H  is  the  simple  structure  transformation  matrix, 
b  is  the  simple  structure  factor  matrix, 

17.6.2b  the  Equations 
3  =  a/  a 


(17.6,1) 
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i 

f 


* 


t  t'  =  S 
Oc  =  at'1 

Y  =  &'  1 
W  «  at'  f 

U  =  Y  1'  -  W 

V  - 

2  -  TVv 

C  e  W  -  Z 

M  =  trl  C 

7  *  M'  M 

,1 

D  = 

H  «  M 

r  =  D  7  D 

b  =  a  H 

17*6.3  Computational  Instructions 


(17-6.2) 

(17.6.3) 

(17.6.4) 

(17.6.5) 

(17.6.6) 

(17.6.7) 

(17.6.8) 

(17.6.9) 

(17.6.10) 

(17.6.11) 
(17.6.12) 

(17.6.13) 

(17.6.14) 

(17.6.15) 

In  this  method  we  may  begin  with 


any  arbitrary  factor  loading  matrix  and  operate  directly  upon  it,  or  we  may 
first  normalise  the  rows  of  the  factor  loading  matrix  as  we  did  in  the  method 
of  Section  17.5.  In  either  case,  we  have  a  binary  hypothesis  matrix  f,  as  in 


the  previous  three  methods. 

Eq.  (17.6.1)  gives  the  initial  computations,  which  consist  of  the  minor 
product  moment  of  the  factor  loading  matrix.  If  this  happens  to  be  a  princi¬ 
pal  axis  matrix,  then,  of  course,  the  matrix  S  on  the  left  will  be  a  diagonal 
matrix. 

We  then  indicate  the  triangular  factoring  of  the  S  matrix  of  Eq.  (17.6.1) 
by  Eq.  (17.6.2). 

The  next  step  is  to  postmultiply  the  factor  loading  matrix  a  by  the  in¬ 
verse  of  the  upper  triangular  factor  of  the  S  matrix  in  Eq.  (17.6.2).  This 
is  indicated  in  Eq.  (17.6.3). 

Next  we  calculate  a  vector  as  indicated  in  Eq.  (17.6.4).  This  is  simply 
a  vector  whose  elements  are  the  sums  of  column  elements  of  the  a  matrix  in 
Eq,  (17.6.3).  It  is,  of  course,  the  transpose  of  the  a  matrix  of  Eq.  (17.6.3) 
postmultiplied  by  a  unit  vector. 

Next  we  calculate  the  minor  product  of  the  a  matrix  of  Eq.  (17.6.3)  by 
the  hypothesis  binary  matrix  f,  as  indicated  in  Eq.  (17*6.5).  This  is  the 
matrix  W  on  the  left  of  Eq.  (17.6. 5). 

We  then  calculate  the  matrix  U  as  indicated  in  Eq.  (17.6*6).  Each 
column  of  the  U  matrix  in  Eq.  (17.6.6)  is  obtained  by  subtracting  from  the 
Y  vector  calculated  in  Eq.  (17.6.4),  the  corresponding  W  vector  frcffl  the 
matrix  calculated  in  Eq.  (17.6.5).  This  is  given  in  matrix  notation  on  the 
right  of  Eq.  (l?.6.6)  as  the  major  product  of  the  Y  vector  of  Eq.  (17.6.4) 
by  a  unit  row  vector  less  the  matrix  W  of  Eq.  (17.6.5). 

We  now  normalize  the  column  vectors  of  U  calculated  in  Eq.  (17.6.6),  as 
indicated  in  Eq,  ( 17.6.7).  We  call  this  the  V  matrix.  The  rl#t  side  of  Eq. 
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(17.6.7)  shows  the  U  matrix  of  Eq.  (17.6.6)  postmultlplied  by  the  reciprocal 
square  root  of  the  diagonal  of  the  minor  product  moment  of  U.  It  is,  of 
course,  not  necessary  to  calculate  the  entire  product  moment  of  U,  but  only 
the  sums  of  squares  of  column  elements,  in  order  to  get  the  D  matrix  used 
in  Eq.  (17.6. 7), 

In  Eq.  (17.6.8)  we  calculate  the  matrix  Z,  which  is  the  V  matrix  of  Eq. 
(17.6.7)  postmultlplied  by  a  diagonal  matrix.  Now  the  diagonal  matrix  is 
made  up  of  the  elements  of  the  diagonal  of  the  minor  product  moment  of  the 

V  matrix  of  Eq.  ( 17.6. 7)  and  the  W  matrix  of  Eq.  ( 17.6. 5).  Here  again,  it 
is  not  necessary  to  calculate  the  minor  product  moment  but  only  the  diagonal 
elements  consisting  of  the  minor  products  of  corresponding  columns  of  the 

V  and  the  W  matrices. 

The  next  step  is  indicated  by  Eq.  (17.6.9).  This  is  the  matrix  W  of 
Eq.  (17.6.5)  minus  the  Z  matrix  of  Eq.  (17.6.8).  This  we  indicate  as  the 
C  matrix. 

Next  we  calculate  the  matrix  M  indicated  in  Eq.  (17.6.IO).  This  is 
obtained  by  premultiplying  the  C  matrix  of  Eq.  (17.6.9)  by  the  inverse  of 
the  upper  triangular  factor  of  the  matrix  S  in  Eq.  (17.6.2). 

We  then  calculate  the  minor  product  moment  7  of  the  matrix  M  calculated 
in  Eq.  (17.6.10),  as  indicated  in  Eq.  ( 17 *6. 11). 

The  next  step  is  to  calculate  a  diagonal  matrix  D,  whose  elements  are 
the  reciprocal  square  roots  of  the  diagonal  elements  of  7  calculated  in  Eq. 
(17.6.11). 

We  calculate  the  transformation  matrix  H  as  indicated  in  Eq.  (17.6.13). 
This  consists  in  normalizing  the  elements  of  the  M  matrix  of  Eq.  (17.6.10) 
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as  shown  on  the  fight  hand  side  of  Eq.  (17.6.13) •  The  M  matrix  is  post- 


multiplied  by  the  reciprocal  square  root  of  the  diagonal  elements  of  its 

the  calculations  for  the  correlations  among  the  primary  reference  axes 
are  indicated  in  Eq.  (17.6.14),  Here  we  pre-  and  postmultiply  the  y_  matrix 
of  Eq.  (17.6.11)  by  the  D  matrix  of  Eq.  (17.6.12). 

Finally,  we  calculate  the  simple  structure  factor  loading  matrix  as  in 
Eq.  (17.6.15).  As  in  previous  methods,  we  postmultiply  the  arbitrary  factor 
loading  matrix  by  the  transformation  matrix  H  calculated  in  Eq.  (17,6.13). 

17.6.4  Numerical  Example 

We  use  the  same  group  centroid  factor  matrix  as  in  the  preceding  section. 

The  tables  for  this  example  will  not  be  discussed  in  detail.  We  merely 
give  below  the  table  number  and  the  corresponding  equation  number  where  such 


an  equation  is  given. 

Table  No. 

17.6.1 

17.6.2 

17.6.3 

17.6.4 

17.6.5 

17-6.6 

17.6.7 

17.6.8 

17.6.9 

17.6.10 
17.6.11 

.  17.6.12 


Equation  No. 

(17.6.1) 

(17.6.2) 


(17.6.5) 

(17.6.6) 

(17.6.7) 

(17.6.8) 

(17.6.9) 

(17.6.10) 
(17.6.11) 
(17.6.13)- 
(17.6.15) 


55 

It  is  of  interest  to  note  that  the  elements  for  each  vector  of  Table 
17 .6,12  corresponding  to  the  zero  element  in  the  corresponding  binary  hypo¬ 
thesis  vector f  actually  do  sum  to  0. 
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fable  17.6,1  -  Matrix  §  =  a/  a 

3.03107  0.56375  0.79188 
0,56375  2.38076  0.37794 
0.79188  0.37794  1.69911 


Table  17.6,2  -  Matrix  t 
1.74100 

0.3353C  1.50610 

0.45484  0.14968  1.21236 


Table  17.6. 3  -  Matrix  t-1 
0.57438 

-0.12787  0.66397 

-0.19970  -0.08197  0.82484 


Table  17.6.4  -  Matrix  W  =  a'  t 

I.59851  0.26077  O.60885 
-0.35654  1.66494  0.21968 
-0.55570  -0.30339  1.59539 


Table  17.6. 5  -  Matrix  U  =  Y  1'  -  ¥ 

mm 

0.86962  2.20735  1.85928 

1.88462  -0.13686  1.30840 

1.29200  1.03969  -0.85909 


Table  17.6.6  -  Matrix  V  =  U  D.' 


0.35569  0.90325  0.76501 
0.77085  -0.05600  0.53835 
0.52846  0.42544  -0.35347 


— fO{l— 
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Table  17,6.7 

-  Matrix 

£  s  V  D  V'W 

0.00003 

0.01195 

0.01538 

0.00006 

0.00004 

-0.00074 

0.00563 

0.01082 

-0.00711 

Table  17.6.8 

-  Matrix 

c  =  w  -  z 

1.59848 

0.24882 

0.59346 

-0.35660 

1.66568 

0.20885 

-0.55574 

-0.30901 

1.60250 

Table  17.6.9 

-  Matrix 

M  =  t,_l  C 

1.07472 

-0.00836 

-0*00586 

-0.19121 

1.13129 

0.00731 

-0.45839 

-0.25489 

I.3218O 

Table  17.6.10 

-  Matrix 

;  L  -  M'  M 

1.40172 

-0.10847 

-0.61559 

-0.10847 

1.34485 

-0.32859 

-O.61359 

-0.32859 

1.74723 

i 

Table  17 .6.11  -  Matrix  H  =  M  D~^ 

-  _ 7 


0.90775 

w*  4>A 

-0.00721 

**  —  v 

-  7 

-0.00443 

-0.16151 

0.97552 

0.00553 

-0.38718 

-0.21979 

0.99997 

Table  17.6.12 

-  Matrix 

b  =  a  H 

0.857 

-0.074 

-0.003 

0.845 

-0.061 

0.029 

0.825 

0.II3 

-0.038 

-0.002 

0.802 

0.077 

-0.028 

0,872 

-0.042 

0.023 

0.853 

-0.023 

-0.028 

-0.012 

0.776 

17 .7  The  Orthogonal  Trans format! on  Matrix 

17.7.1  Characteristics  of  the  Method.  As  indicated  earlier  in  this 
chapter,  it  is  sometimes  desirable  to  impose  the  condition  of  orthonormality 
on  the  simple  structure  transformation  matrix.  This  restriction  can  be  used 
on  any  type  of  arbitrary  factor  loading  matrix  which  is  to  be  transformed  to 
a  simple  structure  hypothesis  matrix.  The  orthonormal  restriction  has  not 
been  generally  used,  however.  Nevertheless,  it  has  a  clear  advantage  over 
oblique  methods  when  applied  to  the  principal  axis  factor  loading  matrix  for 
which  a  simple  structure  binary  hypothesis  matrix  is  available.  Then  the 
simple  structure  factor  score  matrix  whose  solution  we  shall  consider  in  a 
later  chapter  is  an  orthonoxmal  matrix.  This  means  that  the  factor  scores 
are  uncorrelated  in  both  the  unrotated  and  the  rotated  solutions. 

One  characteristic  of  this  method  is  that  it  is  much  more  laborious 
computationally, and  therefore  not  recommended  for  desk  computers.  The  method 
requires  successive  solutions  of  the  basic  structure  of  certain  matrices 
which  are  required  in  the  repeated  approximations  to  the  final  orthonormal 
transformation  matrix, 

17.7.2  Computational  Equations 

17.7.2a  Definition  of  Notation 

a  is  an  arbitrary  factor  matrix, 
f  is  the  binary  hypothesis  matrix. 

r  is  the  correlations  of  the  simple  structure  reference  axes. 
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H  is  the  simple  structure  transformation  matrix, 
b  is  the  simple  structure  factor  matrix. 


17.7.2fc  The  Equations 

7  =  a'  f 

G  =  7Dj/f 

Gi  -  a'  bi-l  "  G 

CiCi  " 

HisCi\  <  Qi 

b  =  a  Hi 

r  =  H{  H,  =  I 

1  i 


(17.7.1) 

(17.7.2) 

(17.7.5) 
(17.7.10 
(17.7*5) 

(17.7.6) 

(17.7.7) 


17.7.5  Computational  Instructions.  The  computational  instructions 
for  this  method  are  brief,  but  the  actual  computations  can  be  lengthy  even 
on  high  speed  computers  if  the  number  of  variables  and  the  number  of  factors 
are  moderately  large,  such  as  those  encountered  in  experimental  investiga¬ 
tions. 

The  method  consists  in  a  set  of  successive  approximation  cycles  begin¬ 
ning  with  any  arbitrary  factor  loading  matrix  which  may  or  may  not  be  normal¬ 
ized  by  rows.  We  have  given  a  binary  hypothesis  matrix,  as  in  the  other 
methods. 


We  'begin  with  Eq.  ( 17. 7.1),  which  gives  the  matrix  7  as  the  product 
of  the  binary  matrix  premultiplied  by  the  transpose  of  the  factor  loading 
matrix* 

We  then  define  a  scaling  of  the  7  matrix  as  in  Eq.  (17.7.2),  the  scal¬ 
ing  diagonal  is  simply  the  diagonal  matrix  of  the  minor  product  moment  of  the 
hypothesis  matrix.  If  this  happens  to  be  one  in  which  there  is  only  a  single 
1  in  each  row,  then  each  diagonal  element  is  the  reciprocal  of  the  number  of 
l's  in  the  corresponding  column  of  the  binary  matrix. 

We  next  indicate  an  iteration  cycle  by  Eq.  (17.7*5)*  Here  we  have  on 
the  right  two  terms.  The  first  of  these  includes  the  preceding  approxima¬ 
tion  to  the  simple  structure  factor  loading  matrix.  This  first  term  is  an 
approximation  to  the  simple  structure  factor  matrix  premultiplied  by  the 
transpose  of  the  arbitrary  matrix.  When  i  =  1  the  approximation  to  the 
simple  structure  matrix,  bQ,  may  be  taken  as  the  simple  structure  matrix 
arrived  at  by  any  one  of  the  four  preceding  methods.  The  second  term  on  the 
right  is  the  G  matrix  calculated  in  Eq.  (17.7*2),  postmultiplied  by  a  diagon¬ 
al  matrix  whose  elements  are  the  diagonal  elements  of  the  minor  product  of 
the  previous  approximation  to  the  simple  structure  factor  matrix  and  the  hy¬ 
pothesis  matrix. 

We  indicate  the  minor  product  moment  of  the  matrix  calculated  in  Eq. 

(17.7.3)  by  Eq.  (l?.7-^> *  Eq.  (17*7*^)  also  indicates  the  basic  structure 

solution  for  this  minor  product  moment.  For  each  approximation  i,  we  calcu- 

2 

late  all  the  vectors  of  the  basic  structure  factors  and  <|, 

Eq.  (17.7.5)  indicates  the  ith  approximation  to  the  orthonormal  trans¬ 
formation  matrix.  This  is  obtained,  as  indicated  on  the  right,  by  multiplying 


from  left  to  right  as  follows:  the  £  matrix  by  the  Q  matrix  by  the  inverse 
of  the  A  matrix  by  the  transpose  of  the  £  matrix. 

Finally,  we  indicate  the  ith  approximation  to  the  simple  structure 
factor  loading  matrix  by  Eqi  (17.7.6).  This  is  the  arbitrary  factor  matrix 
postmultiplied  by  the  ith  approximation  to  the  transformation  matrix  solved 
for  in  Ecu  (17.7.5). 

This  iteration  procedure  continues  until,  according  to  some  criterion, 
the  approximations  are  close  enough.  Presumably,  the  trace  of  the  matrix 
given  in  Eq*  (17.7.4)  would  provide  a  satisfactory  criterion.  When  this 
trace  does  not  change  by  more  than  a  specified  amount  from  one  approximation 
to  another,  we  may  discontinue  the  computations. 

Eq.  (17.7.7)  assumes  that  the  computations  have  stabilized,  and  there¬ 
fore  we  have  the  minor  product  moment  of  the  current  H  or  transformation 
matrix.  This  will  be  a  check  on  the  computations  and  by  definition  this 
should  be  an  identity  matrix. 

17.7.4  Numerical  Example.  We  begin  with  the  same  group  centroid  matrix 
and  binary  hypothesis  matrix  as  in  the  previous  section. 

Tables  17.7.1  and  17.7.2  show  the  computations  indicated  by  Eqs.  (17.7.1) 
and  (17.7.2),  respectively. 

The  remaining  tables  are  for  the  10th  approximation,  as  follows: 

fables  17.7.5  and  17.7.4  give  the  computation  indicated  by  Eq,  (17.7*3). 

Table  17.7.5  gives  the  minor  product  moment  of  the  matrix  C  in  Table 
17.7.4. 

_  i 

Table  17.7.6  gives  the  matrix  (O'  C)  g  =  Q  &  V  which  is  required 
in  fable  17.7.7  and  Eq.  (17.7.5). 


Table  17.7.7  gives  the  approximation  to  the  orthogonal  transformation 
matrix  H  as  indicated  by  Eq.  (17.7.5). 

Table  17.7*8  is  the  approximation  to  the  simple  structure  factor  matrix, 
as  indicated  by  Eq.  (17.7.6). 
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fable  17.7.1 

-  Matrix  7  =  a;  f 

2.78300 

0.45400 

1.06000 

-0.00100 

2.59500 

0.53500 

0.00000 

0.00000 

2.24400 

fable  17.7.2 

-  Matrix  0  =  7  D^,f 

0.92767 

0.15133 

0.35333 

-0.00033 

0.86 500 

0.17833 

0,00000 

0,00000 

0.74800 

Table  17.7.3 

-  Matrix 

a.'  bi  for  i_  =  10 

0.62649 

2.03184 

-2.37365 

2.39956 

0.37041 

-0.50667 

0.56197 

1.82739 

0.04159 

Table  17.7.4 

-  Vector 

D^/  fl  and  Matrix 
i  -1 

O 

H* 

II 

a'  \-l  "  G  ,f 

- - -  l-l 

-0.043 

-0.044 

0.040 

0,66639 

2.03846 

-2.38776 

2.39955 

0.40826 

-0.51379 

0.56197 

1.82739 

0.01172 

Table  17.7.5 

-  Matrix  C'  for  i  =  10 

6.51772 

3.36497 

-2.81748 

3.36497 

7.66132 

-5.05569 

-2.81748 

-5.05569 

5.96551 

Table  17.7.6 

-  Matrix 

ei P 

S’ 

«* 

» 

H 

O 

0.44064  -0.07679  0.06205 
-0.07679  0.50770  0.22263 
0.06205  0.22263  0.58089 


17*8  Mathematical  Proofs 


17.8.1  The  Multiple  Group  Matrix 

Given  the  correlation  matrix  R  and  the  binary  simple  structure  hypo¬ 
thesis  matrix  £,  let 

F  =  R  f  (17.8.1) 


and 


S  =  F'  f 


(17.8.2) 


tt'  =  S 


(17.8.3) 


Then  the  multiple  group  factor  matrix  is  well  known  to  be 

a  =  F  t'*1  (17.8.4) 

Assume  now  we  wish  to  find  the  simple  structure  matrix  b  of  best  fit  to  f. 

We  consider 

a  H  =  b  (17.8.5) 


and 


b  -  f  D  =  e 

where  D  is  diagonal  and  where  for  H  in  Eq.  (17.8.5)  we  have 
%'H  =  1 


(17 .8.6) 


(17.8.7) 


We  wish  to  minimize 
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tr  e'€  .  *  (17.8.8) 

The  solution  is  well  known  to  be 

H  =  (s'  a)"1  a'  f  D  (17.8.9) 

From  Eqs.  (17,8.2),  (17.8.3),  (17.8.4),  and  (17.8.9) 

H  ■>  t'  (F'  F)’1  S  D  (17.8.10) 

Let 

F/  F  =  G  (17.8.11) 

From  Eqs.  (17.8.3),  (17.8.10),  and  (17.8.11) 

H'  H  =  D  S  G*1  8  G"1  S  D  (17.8.12) 

Let 

c  =  g'1  s  (17.8.13) 

and 

g  =  SC  (17.8.14) 

From  Eqs.  (17,8.7),  (17.8.12),  (17.8,13),  and  (17.8.14) 

H'  H  =  DgCD  (17.8.15) 

from  Eq.  (17.8.15) 


(17.8.16) 
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From  las,  (17.8.4),  (17.8.5),  (17.8.11),  anl  (17,8.16) 

b  =  F  C  D~^c  (17.8,17) 

17,8.2  The  Principal  Axis  Matrix 
Given  the  principal  axis  factor  matrix 

a  =  Q  5^  (17.8.18) 

and  the  binary  hypothesis  matrix  f,  consider  the  least  square  transformation 

a  H  =  b  (17.8.19) 

such  that  in 

b  -  f  D  =  e  (17.8.20) 

The  trace  of  €fe  is  minimized  with  a  diagonal  D  such  that  for  H  in  Eq. 
(l7«8,19)  we  have 

dh,  H  =  I  (17.8.21) 

From  Eqs.  (17.8.19)  and  (17.8.20) 

a  H  *  f  D  =  6  (17.8.22) 

The  solution  for  H  is  obviously 

H  -  (a7  a)'1  a'fD  (17,8.25) 

From  Eq.  (17.8,18)  in  Eq,  (17.8.25) 

H  a  6*:  a7  f  D  (17.8.24) 
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b 


(17.8.52) 


b  -  f  D 


=5 


4 9 


(17.8.35) 


with.  H  de-teanined.  S©  'tjbat  t%  §'§  is  minimised  and  D  is  a  diagonal  such  that 


^11  /  g  =  1  ( 17 .8.34) 

Then  the  solution  for  H  is  well  Isnown  to  be 

H  =  (A'  A)4  A'  f  D  (17.8.35) 


M  =  (A'  A)4  A'  f  (17.8.36) 

From  Scj.8,  (17.8.34)  and  (17.8.36) 

D  =  *  (17.8.37) 

and  from  Eqs.  (17.8.35), (17.8.36),  and  (17.8.37) 

H  =  M  D  (17.8.38) 

17.8*4  The  Zero  Partial  Bum  Simple  Structure  Matrix 

Given  the  arbitrary  factor  matrix  a  and  the  binary  hypothesis  matrix  f, 

let 

L  =  11'  -f  (17.8.39) 

be  called  the  supplementary  matrix  to  £ »  Consider 

a  M  e  b  (17.8.40) 
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and 


4  -  f  =  e  (17.8.41) 

with  the  restriction  that 

Vl  »  0  (17.8.42) 

That  is,  the  hypothesized  0's  in  £  sum  to  0  in  b  for  corresponding  columns. 
From  Eqs.  (17.8.40),  (17.8.41),  and  (17.8.42)  we  write 

tr  (6/  e  -  2  D^  Dx)  =  *  (17.8.43) 

where  is  a  diagonal  matrix  of  Lagrangian  multipliers. 

It  will  be  simpler  now  to  consider  Eg..  (17.8.43)  with  respect  to  each 
vector  separately.  From  Eqs,  (17.8.40),  (17.8.41),  and  (17.8.43) 

M'  a'  aM,  -2  M'  a'f.+f'  f.  -2M'  a'  L.X.  =  \|r, 

•  -4  •  !  *1  *1  *1  *1  *1  «X  X  X 

(17.8.44) 

Differentiating  Eg,.  (17.8*45)  symbolically  with  respect  to  M7,  and  equating 

t_X 

to  0  gives 

=  2  (a'  a  M#l  -  a'  f^  -  a'  L#i  X^)  =  0  (17.8.45) 

•  ^ 

From  Eq.  (17.8.45) 

M,i  =  (a'  a f  a'  f  £  +  (a'  a f  a'  \±  (17.8.46) 

Premultiplying  Eq.  (17.8.46)  by  L'±  a  and  using  Eqs.  (17,8.40)  and  (17.8.42) 
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x. 

x 


L%  a  (a'  a)'1  a'  f  . 
L'  a  (s'  a)'*  a'  L  . 

•  1  •  4- 


Let 


(17.8.47) 


t  t7  =  a7  a 


(17.8.48) 


a  =  a  t'*1 


(17.8.49) 


Using  Eqs.  (17.8.48)  and  (17.8.49)  in  Eqs.  (17.8.46)  and  (17.8.47),  we  get, 
respectively, 

M  .  =  t7'1  (a'  f  .  -  a7  L  .  X.)  (17-8.50) 

•  X  •  1  #11 


xi  - 


L7,  a  a1  f  . 

l7.  a  a7  L  , 

•  i  *x 


Using  Eq.  (17.8.51)  in  Eq.  (17.8.50) 


M 


.i 


,  a'  L  .  L7  a  a7  f  . 

t7'1  (a7  f  ,  -  - ~ - —  ) 


»i 


L7  a  a7  L  , 

•  X  •  1 


or 


Let 


M#i  *  t7*1  (1 


a7  L  .  L7,  a: 

•  X_  »_X _ 

L7 .  a  a7  L  . 
tx  *i 


}  *  f,i 


(17.8.51) 


(17.8.52) 


(17.8.55) 


a7  L,  =  U  .  (17.8.54) 

•  X  .1 


(17.8.55) 


Using  las.  (17.8. 58),  (17,8,59),  and  (17.8.60)  in  Bq.  (17.8.57) 

M  =  %•*  (W  -  V  Dv,w)  (17.8.65) 

If  now  we  wish  to  normalize  M  we  have 

H  =  (17.8.66) 

17.8.5  The  Orthogonal  Transformation  Matrix 

Suppose  we  have  a  binary  hypothesis  matrix  f,  and  an  arbitrary  factor 
matrix  a  which  we  wish  to  transform  by  a  square  orthonorraal  transformation 
to  the  best  least  square  approximation  to  f  D,  where  D  is  a  diagonal  to  be 
determined.  We  let 

b  =  a!  (17.8.67) 

where  by  hypothesis 

H;  H  =  I  (17,8.68) 

The  approximation  equation  is 

b  -  f  D  =  e  (17.8.69) 

We  write  the  least  square  function  with  the  constraint  in  Eq.  (17.8,69)  as 
tr  (€'  €  .  H'  H  X)  =  f  (17.8.70) 

where  X  is  a  matrix  of  Lagrangian  multipliers  and 


54 


0 


From  Eqs.  (17*8*67)  and  (17*8. 69)  in  Eq.  (17.8.70),  we  have 

7  =  tr  (H'  a'  a  H  -  2  H'  a'  f  D  +  D  f'  f  D  -  H'  H  X) 

Differentiating  Eq.  (17,8,71)  symbolically  with  respect  to 
ing  to  0  gives 

^-1,  =  2  (a'  a  H  -  a'  f  D  -  H  X)  =  0 

8  H' 

To  differentiate  Eq.  (17*8.71)  with  respect  to  D,  we  let 
VD  *  D1 

Using  Eqs.  (17.8.67)  and  (17*8.74)  in  Eq.  (17.8.72)  gives 

♦  -  tr  (b'  b  -  2  Dfe/f  1  +  v£  Df,f  VD  -  H'  H  x) 

Differentiating  Eq.  (17.8.75)  symbolically  with  respect  to  and 
to  0  gives 


8  Jr 


•  *2  <Vf  1  -  Vf  V  "  0 


From  Eq*  (17*8*76) 


D  *  0j,f  V  f 

Eqs.  (17.8,67)  and  (17*8*77)  in  Eq.  (17*8.73) 


a  b  *  a'  f  t>fl£  Db/f 


let 


(17.8.72) 
and  equat- 

(17.8.73) 

(17*8.74) 

(17*8.75) 

equating 

(17.8.76) 

(17*8,77) 

(17*8.78) 


R  X  s  0 


c  =  P  A  Q'  (17.8.82) 

From  Eqs.  (17.8.81)  and  ( 17.8. 82)  in  Eq.  (17.8.80) 

P  A  Q'  X4  *  H  (17.8.83) 

Because  of  Eqs,  (17.8.68)  and  (17.8.72),  the  only  X^  which  will  satisfy  Eq. 
(17.8.83)  is 

X4  =  Q  A1  Q,1  (17.8.84) 

Therefore,  from  Eqs.  (17.8.83)  and  (17.8.84) 

H  =  P  Q'  (17.8.85) 

Prom  Eq.  (17.8.82)  we  have 

C'  C  *  Q  A2  (17.8.86) 

From  Eq,  (17.8*82) 

f  s  0  ^  A4  (17.8.87) 
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From  Eqs.  (17.8.85)  and  (17,8.87) 


H  =  C  QAJ  Q' 

(17,8.88) 

From  Eqs,  ( 17.8.67)  and  (17.8.88)  we  have 

b  =  a  C  Q  A*1  Of 

(17.8.89) 

We  may  start  with  the  approximation 

bQ  =  a 

(17.8,90) 

Then 

C1  “  a'  a  -  G  Da/f 

(17.8.91) 

ci°i  -  ^4^ 

(17.8.92) 

h  -  ci\4c|i 

(17.8.95) 

In  general. 

C.  *  a'  b,  .  -  G  D,  #  _ 

i  i-1  o^xf 

(17.8.9*0 

C'Ci  .  q^q' 

(17.8.95) 

h  *  Ci\4*% 

(17.8.96) 

bi  =  aHi 

(17.8.97) 

* 


CHAPTER  18 


ANALYTICAL  ROTATIONS 

We  saw  in  Chapter  1?  that  if  we  have  some  hypothesis  as  to  which  vari¬ 
ables  have  high  loadings  and  which  have  low  loadings  in  each  factor,  we  may 
set  up  a  binary  hypothesis  matrix  and,  by  least  square  procedures  with  certain 
constraints  on  the  transformation,  get  the  best  approximation  to  the  binary 
matrix.  We  saw  that  the  transformation  matrices  for  these  procedures  are 
not,  in  general,  square  orthonormal  unless  we  impose  this  additional  con¬ 
straint,  as  in  the  last  method  of  that  chapter.  In  many  cases,  however,  a 
binary  matrix  may  not  be  available,  or  it  may  be  that  the  hypotheses  are 
poorly  satisfied  by  the  data. 

It  is  therefore  desirable  to  have  analytical  methods  which  are  inde¬ 
pendent  of  the  a  priori  hypotheses  of  the  experimenter.  These  analytical 
methods  presumably  should  approximate  the  criteria  outlined  in  the  introduc¬ 
tion  to  Chapter  17.  These  are  the  criteria  formulated  by  Thur stone  (1947). 

A  great  many  methods  have  been  proposed  for  analytical  rotations  to 
simple  structure  factor  loading  matrices.  The  earlier  methods  were  proposed 
by  Thurstone  (194-7),  followed  by  several  methods  developed  by  Horst  (1941) 
and  Tucker  (1944)  which  were  serai -analytical.  Later  Wrigley  and  Newhouse 
(1952)  proposed  more  completely  analytical  procedures.  Then  followed  the 
work  of  Carroll  (1953),  Saunders  (1953), and  several  others.  Perhaps  the 
best  known  methods  are  based  on  the  work  of  Kaiser  (1958). 

In  any  case,  although  the  mathematical  thinking  and  development  which 
have  gone  into  many  of  the  proposed  analytical  methods  is  ingenious,  these 
methods  have  not  resulted  in  the  success  which  may  have  been  hoped  for.  The 


methods  in  general  are  very  laborious  computationally,  even  with  the  high 
speed  computers,  and  often  they  do  not  give  results  which  erne  close  to  sat¬ 
isfying:  the  cKkteri'a  of  simple  structure. 

At  least  two  conditions  should  be  satisfied  by  analytical  methods,  or 
by  any  method  of  rotation.  First,  the  factor  loadings  should  be  relatively 
invariant  with  respect  to  the  group  of  entities  on  which  the  data  are  collected. 
Second,  a  subset  of  factor  loadings  should  be  relatively  invariant,  irrespec¬ 
tive  of  which  particular  battery  of  variables  includes  that  subset.  This 
latter  criterion  is  subject  to  certain  further  qualifications,  but  one  cri¬ 
terion  of  a  good  transformation  procedure  is  that  the  factor  loadings  of 
variables  be  relatively  invariant,  both  with  respect  to  the  sampling  of  en¬ 
tities  and  the  sampling  of  attributes. 

One  of  the  chief  difficulties  with  most  of  the  analytical  methods  which 
have  been  developed  is  that  they  are  greatly  influenced  by  the  particular 
selection  of  variables  which  go  into  the  correlation  matrix.  In  this  book 
we  shall  not  attempt  to  give  an  account  of  all  of  the  analytical  methods 
which  have  been  proposed.  We  shall,  however,  briefly  describe  the  methods 
of  Professor  John  Carroll,  whose  pioneering  and  ingenious  work  may  eventual¬ 
ly  result  in  more  adequate  methods. 

Carroll  (1953)  proposed  that  we  have  a  minimum  number  of  negative  factor 
loadings ,  and  that  such  as  were  present  should  be  small.  His  criterion  for. 
transformation  was  based  on  the  squared  factor  loadings  of  the  transformed 
matrix.  Therefore  the  signs  would  not  influence  the  criterion  of  goodness 
of  transformation.  For  this  matrix  he  required  in  his  early  model  that  the 
minor  products  of  all  possible  pairs  of  vectors  be  a  minimum.  This  meant, 
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,  in  effect,  that  for  any  pair  there  should  he  a  number  of  very  small  squared 

factor  loadings;  there  should  he  very  few  factor  loadings  which  were  high 
for  both  factors;  and  that,  for  those  which  were  high  in  one  factor,  the  load¬ 
ings  should  be  low  in  the  other,  and  vice  versa.  Here,  then,  are  included 
three  of  the  Thur stone  criteria  of  simple  structure.  These  conditions  would, 
of  course,  satisfy  the  criterion  that  the  minor  product  moment  of  the  two 
vectors  of  squared  factor  loadings  should  be  small. 

Carroll  (1953)  worked  out  an  ingenious  computational  procedure  for  the 
high  speed  computer  for  achieving  suck  a  minimum  for  all  pairs  of  factor  load¬ 
ing  vectors.  The  difficulties  with  the  procedure  were,  first,  it  was  strong¬ 
ly  influenced  by  the  particular  variables  in  the  set,  and  second,  it  resulted 
in  too  few  high  factor  loadings  and  too  many  negative  loadings  of  medium  size. 
The  transformation  matrix  in  general  was  such  that  the  correlations  of  the 
*  reference  axes  were  negative,  and  the  correlation  among  the  primary  factors, 

as  discussed  in  Chapter  21,  tended  to  be  positive. 

Later,  Carroll  (1957)  changed  this  criterion  by  considering,  not  the 
minimization  of  sums  of  minor  products  for  all  pairs  of  squared  element  factor 
loading  vectors,  but  rather  the  minimization  of  the  covariance  of  these  vec¬ 
tors,  This  resulted  in  an  overcorrection  for  the  limitations  of  the  previous 
methods . 

The  earlier  of  these  methods  was  called  the  quartimin  method,  and  the 
later  one  was  called  the  eovarimin  method.  Carroll  found  from  empirical  in¬ 
vestigation  that  neither  of  these  methods  worked  very  well.  The  former  pro- 
,  cedure  was  biased  in  favor  of  reference  axes,  which  were  too  low  in  correla¬ 

tion  among  themselves,  and  the  latter  method  resulted  in  reference  axes  which 


were  too  highly  correlated,  For  the  latter  method,  the  large  simple  struc¬ 
ture  factor  loadings  tended  to  he  too  large  and  the  small  ones  tended  to  he 
considerably  greater  than  0. 

Carroll  then  formulated  a  combination  of  the  two  methods  so  as  to  neutra 
lize  the  undesirable  effects  of  both .  This  combination  of  the  quartimin  and 
covarimin  methods  has  been  called  the  ohlimin  method.  The  procedure  still 
left  a  decision  as  to  just  how  to  combine  the  two  procedures,  and  a  certain 
amount  of  arbitrariness  remained. 

The  great  advantage  of  Carroll' s  approach  is  that  one  need  not  hypothe¬ 
size  as  to  whether  a  transformation  is  orthonormal  or  oblique .  The  solution 
itself  purports  to  solve  for  the  correlations  among  the  simple  structure  ref¬ 
erence  axes  and  the  primary  factor  axes.  Unfortunately,  even  with  the  in¬ 
genious  rationale  and  the  extraordinarily  elaborate  computational  procedures 
which  have  been  worked  out,  the  methods  still  have  not  demonstrated  their  use 
fulness  for  some  sets  of  experimental  data. 

Currently,  it  appears  that  the  work  of  Kaiser  (1958)  has  had  more  practi 
cal  impact  on  the  work  of  factor  analysts  than  that  of  other  investigators. 
The  procedures  of  Kaiser  specify  an  orthonormal  transformation.  This  makes 
the  mathematics  and  the  computational  routines  considerably  more  straight¬ 
forward  and  amenable  to  the  application  of  the  basic  structure  concepts .  It 
does  impose  limitations  on  the  results  to  be  expected.  In  particular,  the 
possibility  of  achieving  relative  invariance  of  transformed  factor  loadings 
with  respect  to  both  sample  of  entities  and  sample  of  attributes  is  more  re¬ 
mote  than  if  more  general  transformation  procedures  were  available .  However, 
since  in  this  book  our  emphasis  is  on  practical  application,  we  shall  give 
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otar  major  attention  to  the  methods  developed  by  Kaiser,  and  variations  of 
these. 

18.1  Characteristics  of  the  Methods 

18.1.1  The  Orthonormal  Transformation  Matrix.  The  methods  we  shall 
consider  do  not  admit  of  oblique  transformations .  The  mathematical  models 
on  which  they  are  based,  and  the  computational  routines,  have  the  restriction 
of  orthonorraali ty  built  into  them.  While  even  yet  a  number  of  investigators 
prefer  the  orthonormal  type  of  transformation  as  more  desirable  from  a  philo¬ 
sophical  and  scientific  point  of  view,  the  tendency  seems  to  be  gaining  ground 
to  prefer  the  earlier  objectives  of  Thurstone  in  his  relaxed  oblique  trans¬ 
formations,  which  he  was  able  to  achieve  by  graphical  methods'  and  shrewd 
subjective  Judgment.  Unfortunately,  many  of  his  followers  were  not  able  to 
apply  the  same  ingenious  insights  and  Judgments  in  their  efforts  to  use  the 
non-analytical  graphical  methods. 

It  is  probable  that  the  quest  to  relax  the  orthonormal  transformation 
by  satisfactory  objective  analytical  procedures  will  eventually  triumph.  If 
so,  it  will  probably  result  in  methods  for  objectively  eliminating  from  the 
variables  contributing  to  the  simple  structure  determination,  those  which 
are  most  complex  in  structure  and  which  tend  to  confuse  the  transformation 
attempts . 

18.1.2  The  Optimizing  Function.  With  the  constraint  that  the  trans - 
formation  shall  be  orthonormal,  the  class  of  solutions  we  shall  discuss  all 
consider  a  transformed  matrix  whose  elements  have  been  raised  to  some  even 
power.  This  means  that  the  new  matrix  has  all  positive  elements . 


In  particular,  we  may  consider  the  matrix  of  squared  elements  of  trans¬ 
formed  factor  loadings,  as  did  Carroll.  Our  attention,  however,  is  directed 
to  only  a  single  factor  vector  of  these  positive  elements  at  a  time.  Speci¬ 
fically,  the  optimizing  criterion  which  Kaiser  (1958)  suggested  is  that  the 
variance  of  each  such  vector  of  positive  elements  shall  be  a  maximum. 

Since  the  elements  are  all  positive  by  hypothesis,  the  maximum  for  each 
vector  would  be  achieved  if  all  of  its  elements  were  either  large  or  0.  There¬ 
fore,  in  working  toward  the  maximization  of  this  variance  criterion,  one  tends 
to  reduce  the  intermediate  loadings  to  a  minimum  and  to  maximize  the  number 
of  large  and  small  loadings.  This  again  satisfies  one  of  the  criteria  of 
Thur stone  for  simple  structure,  i.e.,  that  each  factor  should  have  a  relative¬ 
ly  large  number  of  near  vanishing  elements,  a  limited  number  of  very  large 
elements,  with  very  few  elements  of  intermediate  size. 

18.1.3  Iterative  Type  Solutions .  All  of  the  models  considered  in  this 
chapter  differ  essentially  from  most  of  those  in  Chapter  17  in  that  the  solu¬ 
tion  for  the  orthonormal  transformation  matrix  is  arrived  at  by  successive  ap¬ 
proximations.  It  will  be  recalled  that  in  the  last  chapter  only  the  last 
model  required  successive  approximations .  This  is  the  one  in  which  the  re¬ 
striction  of  orthonormality  of  the  transformation  is  imposed. 

The  varimax  solution,  as  it  was  called  by  Kaiser  and  as  developed  by 
him,  consists  of  a  large  number  of  orthonormal  transformations  involving 
only  two  factor  vectors  at  a  time.  The  procedure  in  general  is  to  start  with, 
say,  the  first  two  factor  vectors,  and  transform  them  by  an  orthonormal  trans¬ 
formation  so  that  the  variance  of  their  squared  elements  is  a  maximum.  One 
then  proceeds  with  the  new  first  and  the  third  vectors,  and  applies  another 


orthonormal  transformation  which  satisfies  the  criterion  of  maximum  variance 
of  squared  elements  for  the  transformed  vectors.  This  procedure  continues 
for  all  possible  pairs  until  the  criterion  of  variances  of  squared  factor 
loadings  ceases  to  improve . 

It  can  be  seen  that  this  would  be  an  extremely  laborious  procedure  for 
desk  calculators .  Even  for  the  high  speed  computers  it  can  be  expensive  and 
time -consuming  if  the  matrices  to  be  transformed  are  very  large --for  example, 
of  the  order  of  500  attributes  by  20  or  50  factors.  After  the  varimax  cri¬ 
terion  is  satisfied,  the  method  of  Kaiser  requires  the  product  of  all  of  the 
orthonormal  matrices  involved  in  the  computational  routine.  This  cumulative 
product  gives  the  orthonormal  transf ormati on  which,  when  applied  directly  to 
the  arbitrary  factor  loading  matrix,  yields  a  transformed  matrix  satisfying 
the  varimax  criterion. 

18.1.U  Accumulation  of  Decimal  Error.  The  method  of  Kaiser,  because 
of  the  very  large  number  of  individual  computations  going  into  the  procedure, 
each  of  which  involves  rounding  errors,  is  subject  to  the  accumulation  of 
considerable  decimal  error  if  the  number  of  variables  is  large .  The  methods 
we  shall  outline  use  somewhat  different  approaches  to  achieve  the  varimax 
criterion.  They  do  not  in  general  accumulate  as  much  decimal  error  as  those 
of  Kaiser.  As  a  matter  of  fact,  they  are  self -correcting  with  respect  to 
both  decimal  and  computational  error . 

18.2  Kinds  of  Methods 

We  shall  discuss  four  variations  of  the  methods  proposed  by  Kaiser. 

These  we  may  call  the  successive  factor  varimax,  the  simultaneous  factor 
varimax,  the  successive  factor  general  varimax,  and  the  simultaneous  factor 


general  varimax. 

18.2.1  Successive  Factor  Varimax.  The  successive  factor  varimax  method 
differs  essentially  from  that  of  Kaiser  in  that  we  solve  for  one  factor  vector 
at  a  time,  rather  than  for  all  of  them  simultaneously.  As  each  factor  is 
solved  for,  it  satisfies  the  varimax  criterion  in  that  the  variance  of  the 
sums  of  the  squared  elements  for  a  factor  is  a  maximum.  Having  found  this 
factor,  we  find  another  in  which  the  transformation  vector  is  orthogonal  to 
the  first.  With  this  restriction,  the  variance  of  the  squared  elements  of 
the  next  factor  vector  is  a  maximum.  We  continue  in  this  way  until  we  have 
found  the  last  factor . 

18.2.2  Simultaneous  Factor  Varimax.  In  this  model  we  start  with  an 
approximation  of  some  sort  to  the  simple  structure  matrix  of  factor  loadings. 
We  then  solve  for  a  second  approximation  to  the  factor  loading  matrix  which 
will  satisfy  the  varimax  criterion.  We  thus  proceed  by  successive  approxi¬ 
mations  to  get  factor  loading  matrices  which  will  yield  better  and  better 
approximations  to  the  matrix  which  ultimately  best  satisfies  the  varimax 
criterion. 

The  restriction  for  each  approximation  is  always  that  the  transforma¬ 
tion  matrix  for  that  approximation  is  orthonormal.  The  procedure  therefore 
yields  all  of  the  final  simple  structure  factor  loading  vectors  simultaneous¬ 
ly,  rather  than  one  at  a  time, 

18.2.5  The  Successive  Factor  General  Varimax.  This  method  is  like  the 
successive  factor  varimax,  which  gets  one  factor  at  a  time  and  maximizes  the 
variances  of  the  squared  factor  loadings,  except  that  we  require  that  some 
even  power  of  the  factor  loading  elements  he  positive.  This  even  power  may 
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be  any  ■value  not  less  than  unity.  In  particular,  we  can  require  that  the 
variance  of  the  absolute  values  of  the  factor  loadings  be  maximum. 

18.2.U  Simultaneous  factor  General  Varimax.  This  model  is  similar  to 
the  one  discussed  in  Section  18.2.2  in  that,  by  successive  iterations,  the 
simple  structure  factor  loading  matrix  is  solved  for  by  approximations  to 
all  of  the  factor  vectors  at  one  time.  It  is  similar  to  the  model  discussed 
in  Section  18.2.3  in  that  the  criterion  which  is  maximized  is  a  generaliza¬ 
tion  of  the  variance  of  the  squared  factor  loadings.  Here  again,  we  maxi¬ 
mize  the  variance  of  some  positive  even  power  of  the  factor  loadings,  where 
the  power  is  not  less  than  unity. 

As  will  be  seen  in  the  mathematical  proof,  Section  18.7 one  may  take 
a  positive  even  power  which  in  the  limiting  case  approaches  the  absolute 
value  of  the  factor  loadings.  On  the  other  hand,  one  could  maximize  the 
<  variance  of  the  fourth  powers,  or  the  four-thirds  powers,  or  any  other  powers 

in  which  the  numerator  of  the  exponent  is  even  and  the  denominator  odd  and 
less  than  the  numerator.  It  can  be  seen  that,  if  the  numerator  of  the  ex¬ 
ponent  is  an  even  number  approaching  infinity,  and  the  odd  number  is  always 
one  less  than  the  numerator,  an  element  raised  to  this  power  would  approach 
the  absolute  value  of  the  element. 

18.3  The  Successive  Factor  Varimax  Solution 

18.3.1  Characteristics  of  the  Method.  It  has  been  indicated  that  the 
successive  factor  varimax  solution  does  not  give  the  same  answer  as  the  methou 
of  Kaiser,  in  which  the  transformations  are  made  two  vectors  at  a  time.  Actu- 
*  ally,  in  the  former  case  the  factor  loading  vectors  for  the  transformed  solu¬ 

tion  tend  to  come  out  in  the  order  of  the  variance  of  their  squared  loadings. 
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This  way  not  always  be  the  case,  as  the  final  result  may  depend  somewhat  on 
the  approximation  one  starts  with.  There  is  currently  no  mathematical  proof 
to  indicate  whether,  or  under  what  conditions,  this  might  be  true. 

Perhaps  the  chief  advantage  of  this  method  is  that  the  simple  structure 
factors  tend  to  ccme  out  in  the  order  of  clarity  of  interpretation,  so  that 
one  may  neglect  the  factors  which  appear  later  in  the  solution  if  they  seem 
to  be  too  obscure  or  ambiguous.  The  method  is  different  in  this  respect  from 
Kaiser's,  since  the  ambiguity  of  the  simple  structure  factors  for  his  method 
seems  to  be  spread  over  all  the  factors  approximately  equally. 

18.3.2  Computational  Equations 
18.3.2a  Definition  of  Notation 

^a  is  the  arbitrary  factor  matrix. 

H  is  the  orthogonal  transformation  matrix. 

b  is  the  simple  structure  factor  matrix. 

b^^  is  a  matrix  whose  elements  are  the  second  powers  of  the 
elements  in  b. 

b^  is  a  matrix  whose  elements  are  the  third  powers  of  the 
elements  in  b. 

(18.3.1) 

(18.3.2) 


18. 5* 2b  The  Equations 
a'  1 
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1  L1 

(18.3.3) 

0H.l  =  laL1. 

(18.3.4) 

ib.i  -  iaoH.i 

(18.3.5) 

,*)  1' 
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(18.3.6) 

lU.l  =  la'  1^.1 

(18.3.7) 

iai  =  /iu!i  iu.i 
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iH.i  -  A 

(18.3.9) 

2b.l  =  Ia  lH.l 

(18.3.10) 

(3)  i' 
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n 

(18.3.11) 

SU.l  =  Ia/  SP.l 

(18.3.12) 

a  •  y8"'.i  Ai 

(18.5.13) 

H  =  j£i 
*tl  sai 

(18.3.14) 

8+lb.l  =  I6,  SK.l 

(18.3.15) 
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2a  s  a* 

(18.3.16) 

iV  s  i-lV+b.i-l 

(18.3.17) 

i\  55  ain  iv 

(18.3.18) 

i\,  =  0H.i 

(18.3.19) 

lb.i  =  ia  0®.i 

(18.3.20) 

/,n  l'  b^ 

Ai  =  sfe.i  *  sb.i  n 

(18.3.21) 

sU.i  =  i&/  sP.i 

(18.3.22) 

A  ‘/7X77t 

(18.3.23) 

H  .  JLk 

8  -1  A 

(18,3.24) 

s+lb.l  ■  1*  3H.i 

(18.3.29) 

i+ia  *  ia  -  \i  H!i 

(18.3.26) 

18.3*3  Computational  Instructions.  We  begin  with  an  arbitrary  factor 
loading  matrix  ^a.  Kaiser  has  recommended,  and  the  practice  seems  to  be 
generally  desirable,  that  any  arbitrary  factor  loading  matrix,  before  it  is 
transformed  or  rotated,  should  be  normalized  by  rows.  We  shall  therefore  as- 
same,  in  this  and  the  succeeding  models,  that  the  arbitrary  factor  loading 
matrices  have  been  normalized  by  rows. 


The  rationale  for  selecting  the  first  approximation  to  the  first  trans¬ 
formation  vector  is  as  follows.  We  assume  that  i*f  there  were,  in  the  set  of 
tests  or  measures,  one  which  measured  one  of  the  primary  factors  rather  ac¬ 
curately,  it  would  have  a  relatively  low  correlation  with  the  average  of  all 
the  variables.  Therefore  we  calculate  a  normalized  vector  of  the  average  of 
the  factor  loadings  by  columns,  as  indicated  in  Eq.  (l8.3«l) •  The  right 
hand  side  of  this  equation  gives  in  the  numerator  a  column  vector  whose  ele¬ 
ments  are  the  sums  of  the  column  elements  of  the  factor  loading  matrix  ^a. 

As  can  be  seen,  the  denominator  scalar  of  this  right  hand  term  is  the  square 
root  of  the  minor  product  moment  of  the  vector  in  the  numerator .  Therefore, 
the  W  column  vector  on  the  left  of  Eq.  ( 18.3*1)  is  a  normal  vector. 

In  Eq.  ( 18.3*2)  we  calculate  the  vector  ^V.  This  is  the  factor  loading 
matrix  .a  postmultiplied  by  the  vector  W  of  Eq.  ( 18.3.1).  This  now  gives  a 

■fH» 

vector  of  correlations  of  the  average  of  all  the  tests  with  each  of  the 
measures.  Presumably,  that  variable  which  correlates  lowest  with  this  aver¬ 
age  would  be  a  relatively  pure  measure  of  a  factor. 

We  therefore  look  for  the  lowest  element  in  the  vector  given  by  Eq. 
(18.3.2).  This  is  indicated  in  Eq.  (18.3.3).  We  use  the  subscript  to 
indicate  the  position  of  this  lowest  value. 

We  then  take  the  row  vector  of  the  factor  loading  matrix  as  the 
zero  approximation  to  the  first  vector  of  the  transformation  matrix  H,  as 
indicated  in  Eq.  ( 18.3.4). 

Next  we  postmultiply  the  factor  loading  matrix  by  the  vector  indicated 
in  Eq.  (13. 3.4).  This  gives  the  first  approximation  to  the  first  transformed 
factor  loading  vector,  as  indicated  in  Eq.  (l8.3»5)» 


Then  we  calculate  the  first  approximation  to  a  p  vector,  as  shown  in 
Eq.  (18.3,6).  On  the  right  hand  side  of  the  equation,  the  first  term  is  a 
vector  whose  elements  are  the  cubes  of  the  elements  of  the-  vector  calculated 
in  Eq.  (18.3.5) •  The  second  term  on  the  right  of  Eq.  (18,3.6)  is  the  vector 
calculated  in  Eq.  (18,3. 5), multiplied  by  a  scalar  quantity  which  is  the  aver¬ 
age  of  the  sums  of  squares  of  the  elements  of  the  vector  in  Eq.  ( 18.3.5). 

This  second  vector  is  subtracted  from  the  first. 

Next  we  calculate  the  first  approximation  to  the  U  ,  vector,  as  indi- 
cated  in  Eq.  ( 18.3.7).  This  is  the  transpose  of  the  factor  loading  matrix 
postmultiplied  by  the  vector  calculated  in  Eq.  (18.3.6). 

We  now  calculate  a  scalar  quantity  as  in  Eq.  (18.3.8).  This  is  the 
square  root  of  the  minor  product  moment  of  the  vector  calculated  in  Eq. 
(18.3.7). 

Next  we  calculate  the  first  approximation  to  the  first  transformation 
vector  as  in  Eq.  ( 18.3.9).  The  vector  calculated  in  Eq.  (18.3.7)  is  divided 
by  the  scalar  calculated  in  Eq.  (18.3.8).  We  see,  therefore,  that  the  vector 
calculated  in  Eq,  (18.3.9)  is  a  normal  vector. 

We  now  calculate  the  second  approximation  to  the  transformed  factor  load¬ 
ing  vector  b  As  indicated  in  Eq.  (18. 3. 10),  we  postmultlply  the  arbitrary 
factor  loading  matrix  by  the  vector  calculated  in  Eq.  (18.3.9). 

We  continue  to  calculate  successive  approximations  to  the  first  trans¬ 
formation  vector  H  ^  and  the  first  simple  structure  factor  vector  b^,  as 
indicated  in  Eqs.  (I8.3.II)  through  (18.3.15).  These  equations  are  the  same 
as  Eqs.  (18.3.6)  through  ( 18,3.10),  except  that  the  prescript  of  1  has  been 
changed  to  the  general  subscript  s,  which  means  the  g  approximation. 
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The  stabilization  limit  may  he  based  on  the  scalar  cc  indicated  in  Eqs. 
(l8»3*8)  and  (18.3.13).  When  this  scalar  has  stabilized  to  a  sufficient  de¬ 
gree  of  accuracy,  we  may  assume  that  the  H  ^  vector  is  sufficiently  accurate, 
and  therefore  that  the  b  ^  vector  is  also  sufficiently  accurate. 

We  then  calculate  a  residual  factor  loading  matrix,  as  indicated  in  Eq. 
(18.3.16).  The  first  term  on  the  right  side  is  the  factor  loading  matrix  a 
with  a  prescript  1,  We  use  this  prescript  to  show  that  it  is  the  original 
arbitrary  factor  loading  matrix,  rather  than  some  residual  matrix  derived 
from  it.  The  second  term  on  the  right  of  this  equation  is  the  major  product 
moment  of  the  factor  loading  vector  b  ,  and  the  transformation  vector  H  . . 
This  major  product  is  subtracted  from  the  factor  loading  matrix  to  give  a 
residual  matrix  a. 

4. 

We  are  now  ready  to  begin  the  computations  for  the  second  simple  struc¬ 
ture  factor  vector  b  and  the  second  transformation  vector  H  2«  We  require 
a  first  approximation  to  the  H  0  vector.  This  is  accomplished  by  consider- 
ing  Eq.  (18. 3. 1?) •  Sere  for  the  subscript  i_ we  substitute  2.  On  the  right 
hand  side  of  the  equation  this  gives  as  the  first  term  ^V,  which  we  calcu¬ 
lated  in  Eq.  ( 18.3.2).  To  this  is  added  b  , ,  calculated  in  the  previous 
cycle  of  computations,  to  give  the  vector  V  with  a  prescript  of  2. 

We  now  consider  Eq,  (18.3.IO)  in  which  the  subscript  takes  the  value 
of  2.  This  equation  means  that  we  find  the  smallest  value  in  the  vector  cal¬ 
culated  in  Eq.  (18.3.17)  and  Call  this  the  Lg  position. 

laving  identified  this  position,  we  then  take  as  our  zero  approximation 
to  the  H  2  vector  the  Lg  row  of  the  residual  factor  loading  matrix  ^a  calcu¬ 
lated  in  Eq.  (18.3.16),  This  is  indicated  in  Eq.  (18.3.19). 
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Next  we  calculate  the  first  approximation  to  b  _  as  indicated  in  Eq. 
(18,3.20),  in  which  i  =  2.  The  right  hand  side  of  this  equation  shows  that 
we  postmultiply  the  matrix  calculated  in  Eq.  ( 18,3.16)  by  the  vector  for 
i  =  2  from  It*  (18.3.19). 

* 

For  the  computation  of  the  s_  approximation  to  the  ith  transformation 
vector  II  ,  and  the  s+1  approximation  to  the  ith  factor  loading  vector  b  . , 
we  now  have  the  series  of  equations  (18.3.21)  through  ( 18. 3. 25). 

Eq.  (18.3.26)  shows  the  general  equation  for  calculating  the  i+1  re¬ 
sidual  factor  loading  matrix  i+^a  from  the  ith  residual  factor  loading  matrix 
.a,  the  ith  factor  loading  vector  b  .,  and  the  ith  transformation  vector  H  .. 

18. 3.^  Numerical  Example.  We  shall  use  the  same  numerical  example 
throughout  to  illustrate  the  various  models  in  this  chapter. 

We  begin  with  the  first  three  factors  of  the  principal  axis  factor  load¬ 
ing  matrix  which  are  given  in  Table  18.3.1. 

Table  18.3.2  gives  the  final  matrix  H  which  transforms  the  arbitrary 
matrix  a  to  the  varimax  simple  structure  matrix  b. 

Table  18.3.3  shows  the  final  approximation  to  the  varimax  factor  load¬ 
ing  matrix.  It  can  be  verified  that  the  rows  of  this  matrix  are  normalized. 
If  desired,  they  may  be  scaled  back  to  the  variances  of  the  rows  of  the 
principal  axis  matrix  whose  transpose  is  given  in  fable  18.3.1. 
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fable  18.5*1  -  Transpose  of  Principal  Axis  Factor  Loading  Matrix 


0.717 

0.740 

0,773 

0.556 

0.463 

0,518 

0,640 

0.615 

0,715 

0,493 

0.478 

0.296 

-0.649 

-0.744 

-0,694 

0.080 

O.166 

-0,034 

0.350 

0.322 

0,4o6 

0.068 

0.181 

0,188 

-0.588 

-0.621 

-0.369 

fable  18.3«2  -  Final  Transformation  Matrix  H  for  Successive  Factor 
Varimax  Solution 

0.502  0.702  0*505 

-0.842  0.551  0.098 

0.199  0.475  -O.857 


Table  l8.3«3  -  Final  Varimax  Factor  Matrix  b  for  Successive  Factor 
Varimax  Solution 


0.016 

0,993 

0,118 

0.035 

0.987 

0.154 

0.238 

0.968 

0.077 

0.978 

0.091 

O.I85 

1.000 

0.018 

0.006 

0,995 

0.095 

0.036 

o,157 

0,244 

0.957 

0,051 

0.253 

O.966 

o,390 

0.384 

0.837 

18.4  Simultaneous  Factor  Varimax  Solution 

lS.4.1  Characteristics  of  the  Method,  The  computational  procedures 
in  this  method  are  essentially  different  from  those  of  the  method  just  de¬ 
scribed.  Successive  iterations  are  required,  but  we  iterate  simultaneously 
to  all  of  the  factor  vectors  of  the  b  or  varimax  matrix,  rather  than  getting 
one  vector  at  a  time.  This  method  of  solution  should  give  exactly  the  same 
results,  within  limits  of  decimal  error,  as  Kaiser's  (1959)  computational 
procedure.  It  appears  to  have  the  advantage  that  the  computations  are  self- 
correcting  and  that  it  does  not  accumulate  decimal  error.  The  time  required 
for  the  computations,  as  compared  with  the  Kaiser  method,  has  not  been  ac¬ 
curately  determined^  but  it  appears  that  for  small  matrices  the  Kaiser  method 
may  be  slightly  faster,  and  for  larger  matrices  this  method  may  be  slightly 
faster. 

One  of  the  characteristics  of  the  method  is  that  each  iteration  requires 
the  basic  structure  solution  of  the  matrix  whose  order  is  equal  to  the  number 
of  factors.  For  high  speed  computers,  however,  this  is  not  a  serious  restric¬ 
tion,  since  the  number  of  factors  would  ordinarily  not  be  over  10  or  15  at 
most,  and  available  computer  programs  are  extremely  rapid  for  calculating 
the  basic  structure  factors  of  matrices  of  this  order, 

18*4.2  Computational  Equations 
18.4.2a  Definition  of  Notation 

a,  H,  b,  b^,  4^2  are  ike  same  as  in  Section. 18. 3. 2a. 
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18. 4 , gb  The  Equati ons 

(3)  D1V1° 

i8  -  ib  '  -  ib  -  „ 

(18.4.1) 

1C  '  l8 

(18.4.2) 

i°'  1°  *  i«  ia2  1«' 

(18.4.3) 

2s  *  «1°  1«>  !«' 

(18.4.4) 

2b  ■ 

(18.4,5) 

it 

(3)  V'i6 

.(3  =  ,bu;  -  .b  -4,  h. 

i  i  in 

(18.4.6) 

« 

i°  -  «'  i8 

(18.4.7) 

i°'  i°  *  /  & 

(18.4.8) 

i.iH  ■  «i°  i« /)  t"' 

(18.4.9) 

i+lb  =  a  i+lH 

(18.4.10) 

18.4.3  Computational  Instructions.  We  ‘begin  with  a  first  approximation 
to  the  simple  structure  factor  loading  matrix.  This  could  be  the  normalized 
arbitrary  factor  matrix  itself,  it  may  be  better  to  start  with  a  more  ac¬ 
curate  approximation,  such  as  some  binary  hypothesis  method,  as  discussed  in 
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the  previous  chapter. 

The  first  step  in  the  computational  cycles  is  Indicated  in  Eq,  ( 18.4.1) . 
On  the  right  hand  side  of  this  equation  we  have  as  the  first  term  a  matrix 
whose  elements  are  the  cubes  of  the  elements  of  the  first  approximation  to 
the  b  matrix.  The  second  term  on  the  right  is  obtained  by  postmultiplying 
the  first  approximation  to  the  b  matrix  by  a  diagonal  matrix.  This  diagon¬ 
al  matrix  consists  of  the  diagonal  elements  of  the  minor  product  moment  of 
this  approximation  to  the  b  matrix,  divided  by  n,  the  number  of  variables. 

The  next  step  is  indicated  in  Eq.  (18.4.2).  Here  we  have  the  first  ap¬ 
proximation  to  a  C  matrix  which  is  the  minor  product  moment  obtained  by  post- 
multiplying  the  transpose  of  the  a  factor  loading  matrix  by  the  p  matrix  of 
Eq.  (18.4,1). 

The  next  set  of  computations  is  indicated  by  Eq.  (18.4.3).  We  get  the 
minor  product  moment  of  the  matrix  calculated  in  Eq.  (18.4.2)  and  find  its 
basic  structure  factors  £  and  A_,  as  indicated  on  the  right  hand  side  of 
this  equation. 

We  then  get  the  second  approximation  to  the  transformation  matrix,  as 
indicated  in  Eq.  (l8,4.4).  Here  we  postmultlply  the  C  matrix  of  Eq.  (18.4.2) 
successively  by  the  factors  Q,  A~~  ,  and  Qf .  This  second  approximation  to 
the  H  matrix  is  now  orthonormal. 

The  second  approximation  to  the  b  matrix  is  given  in  Eq.  ( 18.4.5).  This 
is  the  factor  loading  matrix  a  postmultiplied  by  the  matrix  of  Eq.  (18.4.4). 

The  general  equations  for  the  i+1  approximations  to  the  H  and  the  b 
matrices  are  given  in  Bqe.  (18,4,6)  through  ( 18.4.10),  which  are  analogous 
to  Eqs,  ( 18.4,1)  through  ( 18.4,5) ,  The  subscripts  1  and  2  have  been  replaced 
by  i  and  i+1. 


18.4.4  Numerical  Example.  Sfe  begin  with  the  same  factor  loading 
matrix  in  this  numerical  example  as  in  the  previous  one, 

Table  18.4.1  gives  the  final  approximation  to  the  varimax  transforma¬ 
tion  matrix  H.  Intermediate  approximations  are  not  given,  although  they 
could  be  readily  outputed  from  corresponding  Fortran  program. 

Table  18.4.2  gives  the  final  approximation  to  the  varimax  factor  load¬ 
ing  matrix.  Here,  too,  the  output! ng  of  intermediate  approximations  may  be 
readily  inserted  in  the  Fortran  program.  It  can  be  seen  that,  aside  from 
the  order  of  the  factors,  the  loadings  do  not  differ  markedly  from  those  in 
the  previous  section.  With  other  data  the  results  may  differ  more  for  the 
two  methods.  As  in  the  previous  section,  the  matrix  is  normal  by  rows. 
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Table  18.4.1  -  Final  Approximation  to  Transformation  Matrix  H 
for  Simultaneous  Factor  Varimax  Solution 

o.6;64  B.o»4rf“2 
0.514  0,851  •  rnM§% 

0.545  -0.220  lH 


Table  18.4,2  -  Final  Approximation  to  Varimax  Factor  Matrix  b 
for  Simultaneous  Factor  Varimax  Solution 

0.980  0.001  0.197 


0.972  -0,017 

0,234 

V  •  ©vi  •*  V  • 

-O.OSii-  0.969 

-0.233 

-0.026  0.998 

•0 .0|9 

-0.100  0,991 

n  i4CA  a  1  n  )i 

-0.085 

U  •  JLOO  -U  •  J.J44 

0.175  -0.008 

0.984 

0.318  -0.351 

0.881 

18,5  The  Successive  Factor  General  Varimax 

18.5.1  Characteristics  of  the  Method.  This  method  is  like  the  one  dis¬ 
cussed  in  Section  18. 3,  except  that  now  instead  of  maximizing  the  variance 

of  the  squared  factor  vector  elements,  we  solve  for  one  vector  at  a  time  so 
as  to  maximize  the  variance  of  some  power  of  the  elements  in  which  the  power 
is  the  ratio  of  an  even  number  to  a  smaller  odd  number.  The  computations 
are  essentially  the  same  as  for  Section  18.3  with  the  difference  that,  hav¬ 
ing  chosen  a  particular  power,  we  have  the  problem  of  finding  the  required 
powers  of  elements,  either  by  means  of  tables  of  logarithmic  and  exponential 
functions  or  by  means  of  computer  program  statements, 

18.5.2  Computational  aquations 

18.5.2a  Definition  of  Notation 

1a,  b,  and  H  are  the  same  as  in  Section  18.3,2a. 


18.5.2b  The  Equations 


so 


s+lb.l  *  la  sK.l 


ia  ■  b,i  n:i 


.  ,.a  s  .a  -  b  .  H' . 
l+l  i  .1  .1 


18.5*5) 
(18,5,6) 
.8,5,7) 


18,5*3  Computational  Instructions.  In  this  model  we  may  let  the  power 


of  the  transformed  factor  loading  elements  he  any  positive  number  k  greater 


than  unity  which  may  he  expressed  as  the  ratio  of  an  even  to  an  odd  integer. 

We  begin  with  Eq.  ( 18,5.1).  Here  we  indicate  the  s  approximation  to  a 
[3  vector,  which  is  analogous  to  the  £  vector  which  we  calculated  in  Section 
18.3.  We  assume  that  some  approximation  to  the  transformed  vector  is  avail¬ 


able.  As  a  matter  of  fact,  we  can  use  the  methods  of  Section  18.3  to  get  this 
approximation.  V/e  raise  the  elements  to  the  2k-l  power  to  get  the  first 
terra  on  the  right  of  Eq,  (l8,5.l),  in  which  the  subscript  s  takes  the  value 
1.  The  second  term  on  the  right  consists  of  the  approximation  vector  with 
elements  raised  to  the  k-1  power  and  multiplied  by  a  scalar  which  is  the  mean 
of  the  kth  power  elements  of  the  vector. 

Eq.  (18.5.2)  is  obtained  by  postraultiplying  the  transpose  of  the  arbit¬ 
rary  factor  loading  matrix  by  the  s  approximation  to  the  0  »  vector  to  give 


an  a  approximation  to  a  U 


,JL 


vector. 


A  scalar  quantity  is  then  calculated  as  in  Eq»  (18.5,3)#  which  is  the 


square  root  of  the  minor  product  moment  of  the  vector  calculated  in  Eq. 
(18.5.2). 


Si 


The  vector  calculated  in  Eq.  (18.5.2)  is  divided  by  the  scalar  calcu¬ 
lated  in  Eq.  (18.5.3)  to  give  the  s  approximation  to  the  first  transforma¬ 
tion  vector  of'  g,  as  indicated,  in  Eq.  ( 18.5.4). 

The  s+1  approximation  to  the  first  factor  loading  b  vector  is  given  by 
Eq.  (18.5.5).  This  is  the  factor  loading  matrix  postmultiplied  by  the  vector 
calculated  in  Eq.  (18.5.4).  When  the  iterations  stabilize  sufficiently,  as 
indicated  by  sane  tolerance  limit  set  on  the  at  scalar  of  Eq.  (18,5,3),  ve 
may  take  the  resulting  b  approximation  as  the  first  simple  structure  factor 
vector. 

We  then  calculate  a  residual  matrix  as  in  Eq.  (18.5,6).  This  is  ob¬ 
tained  by  subtracting  the  major  product  of  the  first  simple  structure  factor 
loading  vector  and  the  corresponding  transformation  vector  H  from  the  arbit¬ 
rary  factor  matrix  a. 

We  now  operate  on  this  new  matrix  with  Eqs.  (18.5.I)  through  (18,5.5) 
in  exactly  the  same  way  as  we  did  on  the  original  matrix.  To  get  a  first 
approximation  we  may  use  the  method  of  Section  I8.3. 

The  general  equation  for  the  i+1  residual  of  the  arbitrary  factor  load¬ 
ing  matrix  a  is  given  by  Eq.  ( 18.5.7). 

18.5.4  Numerical  Example.  We  begin  with  the  same  data  as  in  Section 
18.3,4  and  take  k  =  — .  Here  We  indicate  only  the  results  of  the  method. 

The  intermediate  computations  are  not  given. 

Table  18. 5.1  is  the  final  approximation  to  the  orthonormal  transforma¬ 
tion  matrix. 

Table  18.5.2  is  the  final  approximation  to  the  varimax  matrix  b.  It 
must  be  remembered  that  this  is  the  transformation  such  that  the  variance  of 


the  ~  power  of  the  elements  in  the  b  matrix  is  a  maximum  for  each  column- - 
with  the  restriction,  of  course,  that  the  columns  were  obtained  one  at  a 
time.  It  is  of  interest  to  compare  this  matrix  with  Table'  18.3*3  of  Section 
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Table  18. 5*1  -  Pinal  Approximation  to  Transformation  Matrix 

H  for  Successive  Factor  General  Varimax  Method 

with  k  = 

—  3 

O.T35  0.412  0.538 

0.089  -0.846  0.526 

-0.672  0.339  0.659 


Table. 18.5.2 


Pinal  Approximation  to  Varimax  Factor  Matrix 


b  for  Successive  Factor  General  Varimax  Method 


with  k 


10 

3 


0.358 

-0.003- 

0.934 

0.395 

0.010 

0.919 

0.349 

0.223 

0.910 

0.356 

0.934 

0.003 

0.171 

0.985 

-0.026 

0.218 

0.975 

0.042 

1.000 

-0.004 

-0.001 

0.99.4 

-0.110 

0.010 

0.957 

0.246 

0.154 

18.6  Simultaneous  Factor  General  Varimax 

18.6.1  Characteristics  of  the  Method.  This  method  is  the  same  as  the 
one  discussed  in  Section. l8.4,  except  that  now  we  maximize  the  variance  of 
any  even  power  of  the  transformed  elements  we  wish,  as  long  as  the  power  is 
greater  than  1.  The  method,  like  that  of  Kaiser  (1958),  solves  simultaneous¬ 
ly  for  all  of  the  transformed  simple  structure  vectors,  rather  than  for  one 
at  a  time.  It  has  the  advantage  that  decimal  error  is  not  accumulated  as  it 
is  in  Kaiser's  method.  It  has  the  disadvantage  that,  if  the  powers  are  not 
whole  numbers,  either  tables  or  library  programs  for  computers  must  be  used 
in  calculating  these  powers.  However,  this  is  not  a  serious  disadvantage 
with  the  computers,  since  simple  Fortran  statements  may  be  written  for  any 
specified  power. 

18.6.2  Computational  Equations 

18.6.2a  Definition  of  Rotation 

a,  b,  and  H  are  the  same  as  in  Section  18.3 • 2a. 
gb  is  the  a  approximation  to  b. 

b(2k-l)  a  ma^rix  Whose  elements  are  those  of  b  raised  to  the 
s  s 

2k -1  power. 

gH  is  the  8  approximation  to  the  transformation  matrix. 
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18.6.2b  The  Equations 

a  _  b(2k-l)  b(k-l)  Pl/3b(k) 

sH  s  s  n 


C  =  a'  p 
s  sK 


C'  C  =  Q  A2  0; 
s  s  s  s^ 


SH  =  ((gC  SQ)  3^)  SQ' 


(18.6.1) 

(18.6.2) 

(18.6.5) 

(18.6.4) 


s+1 


a  _H 


(18.6.5) 


18.6.3  Computational  Instructions.  In  the  example  of  the  model  dis¬ 
cussed  here,  we  shall  again  assume  that  it  is  the  variance  of  the  kth  power 
of  the  elements  in  the  transformed  factor  vectors  which  we  wish  to  maximize. 

V/e  may  begin  with  the  arbitrary  factor  loading  matrix  itself,  presumably 
normalized  by  rows.  We  consider  Eq.  (18.6.1)  for  the  prescript  s  equal  to  1. 
The  first  term  on  the  right  is  a  matrix  whose  elements  are  the  2k -1  powers 
of  the  elements  in  ^b.  The  second  term  on  the  right  has  for  the  first  factor 
a  matrix  whose  elements  are  the  k-1  powers  of  the  elements  in  b.  This  matrix 
is  postmultlplied  by  a  diagonal  matrix  whose  elements  are  the  means  of  col¬ 
umns  of  a  matrix  obtained  by  raising  the  elements  of  b  to  the  kth  power. 

Eq.  (18.6.2)  indicates  the  computation  of  a  matrix  C.  This  is  the  trans¬ 
pose  of  the  arbitrary  factor  matrix,  postmultlplied  by  the  |3  matrix  of  Eq. 
(18.6.1). 

Eq.  (18.6.3)  is  the  minor  product  moment  of  the  matrix  obtained  in  Eq. 
(18.6.2),  and  also  indicates  the  solution  for  the  basic  structure  factors, 
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Q  and  A_,  of  this  matrix.  This  involves,  as  in  Section  18.4,  basic  structure 
computations  outlined  in  earlier  chapters. 

The  calculation  of  the  £  approximation  to  the  H  matrix  is  given  in  Eq. 
(18.6.4).  This  is  obtained  by  postmultiplying  in  turn  the  C  matrix  of  Eq, 
(18.6.2)  by  the  Q  matrix  shown  in  Eq.  (18.6.3),  by  the  inverse  of  the  A 
matrix  of  Eq.  (18.6.3),  by  the  transpose  of  the  Q  matrix  of  Eq.  (18.6.3). 

The  s+1  approximation  to  the  transformed  factor  loading  matrix  is  given 
in  Eq.  (18.6.5).  This  is  the  arbitrary  factor  loading  matrix  a  multiplied 
by  the  matrix  of  Eq.  (18.6.4).  A  stabilization  limit  may  be  set  on  the  sum 
of  the  elements  of  the  basic  diagonal  in  Eq.  (18.6.3).  These  will  in  gen¬ 
eral  increase  asymptotically  to  an  upper  limit. 

18.6.4  Numerical  Example.  We  begin  with  the  same  principal  axis  matrix 
as  in  the  preceding  sections. 

Table  18.6.1  gives  the  final  approximation  for  the  transformation  matrix 
H. 

Table  18.6.2  gives  the  final  approximation  for  the  varimax  factor  matrix 
b.  The  factors  can  readily  be  identified  with  those  from  previous  solutions 
in  this  chapter,  even  though  the  results  expectedly  differ  by  more  them  deci¬ 
mal  accuracy. 
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Table  18.6.1  -  Final  Approximation  to  Transformation  Matrix  H 

for  the  Simultaneous  Factor  General  Varimax 

Solution  with  k  =  r— 

3 

0.670  -0.476  -O.569 
0.513  O.85I  -0.108 

0.336  -0.220  0.815 


Table  18.6..2  -  Final  Approximation  to  the  Varimax  Factor  Matrix 
b  for  the  Simultaneous  Factor  General  Varimax 
Solution  with  k  =  ^ 


O..982  0.002  0.188 
0,974  -0.017  0.224 
0.963  -0.223  0.153 
-0.089  0.971  -0.223 
-0.028  0.999  -0.040 
-0.104  0.992  -0.076 
0,177  -0,123  0.976 
0.185  -0.01?  0,983 
0.328  -0.358  0.874 
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18.7  Mathematical  Proofs 

18.7.1  Successive  Factor  Varimax 
let 

b  =  a  H  (18.7.1) 

(p) 

and  bv  '  be  a  matrix  whose  elements  are  the  squares  of  those  in  b.  Consider 
the  function 


max 


(18.7.2) 


This  is  the  well  known  varimax  criterion  of  Kaiser  (1958)  which  maximizes 
the  variance  of  the  squared  factor  loading  vectors  in  b.  We  impose  the  re¬ 
striction  that 

H'  H  .  *  1  (18.7.5) 

•  X  il 

We  let 

D,  1  =  h  ,  (18.7.M 

b.l  -1 

From  Eqs.  (18.7.1),  (18.7.2),  (18.7.5),  and  (18.7.4)  we  write 

*1  ■  H'.i  »'  “b  <*  •  ¥'>  \ .  •  “.1  -  H'.i  B.i  H  (l8-7-5) 

•  X  iX 

where  X^  is  a  Lagrangian  scalar. 

Differentiating  Eq.  (18.7.5)  symbolically  with  respect  to  and  equat 
ing  to  0,  we  have 


2  ((a' 


1J.' 

n 


)  “b  ■ 

•  X 


H 


.1 


-  H.i  h] 


0 


(18.7.6) 
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From  Eqs.  ( 10. 7.1)  and  (18.7.6) 

<  fk<5)  .  „  , .”'.1  Li 


*'■  <*3 


.1  n 


>  -  H.iS 


Let 


p  =  b^)  -  b  .  ■ 

•  *1  *1  «1  n 


From  Eqs.  ( 18.7.7)  and  ( 18.7.8) 
a'  *.l  =  H.l  X1 

We  start  with  a  first  approximation  by  considering,  say, 
s'  1 


W 


,/V  a  a'  1 
:  a  W 


and  find  the  smallest  element  in  ^V,  say  We  may  then  let 


o\i  -  *V 


We  have  as  the  general  iteration  equations 

Sbll  8b.l 


sP.l  • 

b'<'  -  b 

S  .1  .8 

a'  s*.l 

-  sU.l 

SH.l  * 

SU.l 

/  W  if 

vs  ,1  s.l 

6+lb.l 

*=  a  H  , 

8  .1 

(18.7.8) 


(18.7.8) 


(18.7.9) 


(18.7.10) 

(18.7.11) 

(18.7.12) 

(18.7.13) 
(18.7.1*0 
(18.7.15) 

.7. 
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From  Eqs.  (18.7,1)  and  (18.7.6) 


a'  {if'V  -  b  -ii—ii 

•  X  •  X  Q 

)  -  HaX1 

(18.7.8) 

Let 

a  b(3)  b  bib4 

.  p.l  .1  .1  n 

(18.7.8) 

From  Eqs.  (18.7.7)  and  (18.7.8) 

a'  P.1  H.l  X1 

(18.7.9) 

We  start  with  a  first  approximation  by  considering,  say. 

a'  1 

,/V  a  a'  1 

(18.7.10) 

jV  »  a  W 

(18.7.11) 

and  find  the  smallest  element  in 

^V,  say  We  may  then  let 

04.1  *  **l. 

(18.7.12) 

We  have  as  the  general  iteration  equations 

Ax  ■  ."S’  •  ,b.i 

8b[l  sb.l 

(18.7.13) 

n 

a'  V.1  -  sU.l 

(18.7.1k) 

Su.l 

8  #1  7u'  U  . 

'/  8  elS  «X 

(18.7.15) 

_,,b  ,  =  a  H  . 

8+1  .1  s  .1 

(18.7.16) 
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To  get  H  -  we  require  that 

H'  H  .  =  0  (18.7.17) 

•  d  •  i. 

H'  H  0  =  1  (18.7.18) 

•  d  •c. 

We  could  then  write 


= 


2  H'.2  H.l  *12  ■  H/.2  H.2  >2 


(18.7.19) 


where  0g  is  analogous  to  Eq.  (18.7.2),  and  where  X12  and  Xg  are  Lagrangian 
multipliers. 

Differentiating  Eq.  (18.7.19)  symbolically  with  respect  to  H'g  and 
equating  to  0  gives,  after  solving  for  X^g  by  means  of  Eq.  (18.7.17), 


-  H.l  H'.l>  *'  p.2  - 


h.2  ^2 


(18.7.20) 


where 


B  _  b(3)  .  b 

P.2  "  °.2  .2  n 


But  from  Eq.  (l8.7*l) 


(1  -“.1  »:i> »'  - -  a.i  Kz 


(18.7.21) 


(18.7.22) 


If  we  let 


2a  =  a  -  bpl  E'^ 


we  may  write  Eq.  ( 18.7.20) 


2&/  =  ^,2  ^2 


(18.7.23) 


(18.7.2^) 
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We  require  that 


H7  H  =  I 


(18.7.3*0 


From  Eqs.  (18.7.32),  (18.7.33),  and  (l8.7.3*0  we  write 

♦i  ■  H'.i  “b., (I  ■  ¥'>  *  h.i  -  b:i  H.t  ‘u +  j,  su  \s  ht 

(18.7.35) 

where  the  X.  .  are  Lagrangian  multipliers, 

jy. 


(3)  b,i  b  i 

P  ,  =  -  b  . 

•  x  •  x  •  i  n 


(18.7.36) 


Differentiating  Eq.  (18.7.35)  symbolically  with  respect  to  H7^,  equating  to 
0,  and  using  Eq,  (18.7.36),  we  have 


a7  -  H  \  ±  =  0 


(18.7.37) 


Or,  setting  up  the  complete  matrix, 


a7  p  -  H  X  =  0 


(18.7.38) 


where  now  X  Is  a  matrix  of  Lagrangian  multipliers,  and  where  because  of  Eq. 
(18.7.35), 


X7  =  X 


(18.7.39) 


and  from  Eq.  (18,7.36) 


P  =  b^)  -  b  •  b 

n 


(18.7.40) 
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« 

„  Fran  Eq.  (18.7.38) 

a'  PX’1  =  H 
Let 


(18.7.41) 


a'  0  =  PAQ' 


(18.7.42) 


be  the  basic  structure  of  e.1  p. 
From  Eq.  (18.7.42) 


PAQ'  X1  =  H 


(18.7.43) 


Because  of  Eqs.  (18.7.34)  and  (l8.7.39)>  the  only  which  will  satisfy  Eq, 
(18.7.43)  is 

X1  =  Q  A*1  Q#  (18.7.44) 

Fran  Eqs.  (18.7.43)  and  (18.7.44) 

H  =  PQ'  (18.7.45) 

We  let 

C  =  a'  0  (18.7.46) 

From  Eqs.  (18.7.42)  and  (18.7.46) 

Q  A2  Q'  =  C..  C  (18.7.47) 

From  Eqs.  (18.7.42)  and  (18.7.46) 
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♦ 


>  - 

P  *  C  Q  A3, 

(18.7.48) 

From  Eq®.  (18.7.45)  and  (18.7.48) 

H  =  CQ^Q' 

(18.7.49) 

The  iteration  procedure  is  as  follows. 

Given  the  ith  approximation  to 

b,  then 

1S  -  I*1'5’  -lb  ^  H 

(18.7.50) 

iC  "  ^  iP 

(18.7.51) 

i*  /  ±Q'  =  ,0'  tC 

(18.7.52) 

> 

i+lH  "  iC  iQ  /  iQ' 

(18.7.55) 

i+lb  =  a  i+lH 

(18.7.54) 

18.7.3  Successive  Factor  General  Varimax 
Let 


* 


b  n  =  a  H  n 


2  », 


k  = 


2  ”2-l 


(18.7.55) 


(18.7.56) 


where  and  rag  are  integers  and  <  m^. 

elements  are  the  kth  power  of  those  in  b  . . 

"*  *1 

•  “ — 

elements  of  bv . '  are  nonnegative. 

»x 

Consider 


't1  -  ¥')  ^ 


Let  b^  be  the  vector  whose 
Because  of  Eq.  (18.7.56)  all 


max 


(18.7.57) 
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with  the  constraint 

H',  H  ,  =  1  (18.7.58) 

•  X  i-L 

t 

Let 

if.  =  0.  ■  k  H',  H  .  L  (18.7.59) 

where  X^  is  a  Lagrangian  scalar. 

Prom  Eqs.  (18.7*55)  and  (18.7.57) 

"'.i*'  ^ (I  -  ¥')  ■  »i  (16-7-6o) 

•  X  •  x 

From  Eqs.  (18.7.59)  and  ( 18.7.60)  we  may  write  the  maximizing  equation 


a 


Sr  -  *  M.' ^  <i  -  V7  °T  *  B.i  ‘ 

*1  .x 


1  1' X  Jt-1 


H  ,  -  X.J  =  0 


(18.7.61) 


We  let 


P.1  =  *.l 


l'  b^ 

(2k-l)  Jk-l)  .1 


.1 


(18.7.62) 


From  Eqs.  (l8.7.6l)  and  (18.7.62) 
a'  0 


.1 


H.l  X1 


(18.7.63) 


We  may  begin  with  sane  arbitrary  b  say,  ^b^,  and  set  up  the  iterative 
equations 

.  .  17  bW 

(18.7.64) 


,  i/  b^ 

_  (2k-l)  _  b(k-l)  s  .1 

sP.l  "  s  .1  sb.l  n 


*'  .P.1  =  3U.l 


(18.7.65) 
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A  ■  jTTT.x 

(18.7.66) 

H  *** 

JL  -  «  •mm  — 

r-1 

(18.7.67) 

Bb.l  *  “8“.l 

(18.7.68) 

after 

stabilizes,  we  calculate 

2a  ■  a  "  H'i 

(18.7.69) 

This  is  then  substituted  for  a  in  Eqs.  (18.7.65)  and  (18.7.68) 

The  general  equation  for  Eq.  (18.7.69)  is,  of  course, 

to  get  b  g. 

i+la  "  ia  '  \i  H'.i 

(18.7.70) 

To  calculate  the  elements  in  b  .  raised  to  the  respective  powers,  we 

proceed  as  follows.  We  let  B  be  any  element  of  b  Now  because  of  Eq. 

“  P  il 

(18.7.56) 

2  (n^  -  m2)  +  1 

2  n>2  "  1 

(18.7.71) 

and 

2  (2  m.  -  m  )  +  1 

2  k  -  1  a  —  •  1  d 

2  m  -  1 

(18.7.72) 

Fran  Eq.  (18.7.71)  all  B^1  and  B2^"1  have  the  same  sign  as  B.  From 
Eq.  (18.7.56)  all  B^  are  positive.  Hence  to  calculate  the  several  required 
powers  of  B  we  have 


Bk  =  antilog  (k  log  |b|  ] 


(18.7.75) 
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Bk"1  *  antilog  [(k  -  l)  log  Ifij]  lBl 

(18.7.7*0 

'  B 

B^"1  =  antilog  [(2  k  -  l)  log  |b|]  i|L 

(18.7.75) 

These  three  powers  of  B  may  be  readily  calculated  from  tables 

or  from  standard 

library  programs  for  computer  installations. 

18.7 .4  Simultaneous  Factor  General  Varimax 

Let 

b  =  a  H 

(18.7.76) 

2  m. 

k  =  —■  — 

2  mg  -  ! 

(18.7.77) 

where  m1  and  are  Integers  and  mg  <  m^.  Consider,  then,  the  generalization 
of  Kaiser's  criterion 


tr  |_b^k)'  (I  -  L±')  b^j  =  max  =  0 

(18.7.78) 

with  the  constraint 

H'  H  =  I 

(18.7.79) 

Fran  Eqs.  (18.7.76),  (18.7.77),  and  (18.7.78) 

2h:i  ••  ■ ' 

•  X 

(18.7.80) 

Consider 

if  =  0  -  k  tr  (H'  H  \) 

(18.7.81) 

By  the  methods  of  the  previous  section,  we  may  let  the  basic  structure  of 
af  P  be 

a'  p  =  P  A  Q'  (18.7.86) 


From  Eq.  (18.7*85) 

a'  P  \l  *  H  (18.7.87) 

To  satisfy  Eqs.  (18.7.87)  and  (18.7.79) 

Q  =  a'  p  Q  A1  Q'  (18.7.88) 


We  may  begin  with  some  arbitrary  approximation  to  b  in  Eq.  (l8.7;T6), 
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say,  jb.  From  jb  and  Eq.  (18.7.83)  we  calculate  Then  from  Eq.  (18.7.86) 


a'  £  -  £  ,Q' 

(18.7.89) 

and  from  Eq.  (18.7.88) 

II 

(18.7.90) 

In  general,  then, 

1/  >,(&) 

P  -  v(2k-l)  Jk-l)  1  sD.i 

sp.i  ■  sD.i  sD.i  n 

(18.7.91) 

a'  s*  "  sP  s*  sQ' 

(18.7.92) 

SH  -  a'  s*  sQ  s** 

(18.7.93) 

s+lb  -  \sH 

(18.7.9M 

The  several  required  powers  of  the  b  .  in  Eq.  (18.7.91)  can  be  calculated 

s  >1 

as  in  the  previous  section. 


CHAPTER  19 


DIRECT  VARIMAX  SOLUTIONS 

We  saw  In  Chapter  18  that  we  may  begin  with  any  arbitrary  factor  load¬ 
ing  matrix,  and  transform  it  to  a  simple  structure  varimax  factor  loading 
matrix  by  means  of  a  square  orthonormal  transformation.  In  this  chapter, 
we  shall  see  how  we  may  avoid  the  intermediate  step  of  first  calculating  an 
arbitrary  factor  loading  matrix,  such  as  in  the  principal  axis,  multiple 
group,  group  centroid,  and  variations  of  these  methods. 

19.1  Characteristics  of  the  Method 

19.1.1  No  Arbitrary  Factor  Matrices  Required.  In  the  procedures  out¬ 
lined  in  the  following  sections,  we  shall  see  how  we  may  operate  directly  on 
the  correlation  matrices,  or  the  score  matrices  from  which  they  are  derived. 
Strangely  enough,  this  seems  to  be  a  novel  approach  for  factor  analysts. 

The  tradition  has  been  to  calculate  first  some  arbitrary  factor  loading 
matrix,  such  as  the  principal  axis,  centroid,  or  other  type  of  arbitrary 
factor  loading  matrix,  whose  major  product  moment  gives  a  reasonable  approxi¬ 
mation  to  the  correlation  matrix.  These  arbitrary  matrices  are  then  trans¬ 
formed  by  the  procedures  outlined  in  the  previous  chapter  or  by  other  analy¬ 
tical,  semianalytical,  or  graphical  methods.  Actually,  the  transition  from 
operations  upon  the  arbitrary  matrices  to  achieve  simple  structure  matrices, 
to  that  of  a  direct  solution  for  the  simple  structure  matrices  from  the  cor¬ 
relation  or  data  matrices,  is  perfectly  natural,  both  from  a  logical  and  a 
mathematical  point  of  view. 

19.1.2  Analytical  Methods.  As  a  matter  of  fact,  however,  the  methods 
of  direct  solution  for  simple  structure  factor  matrices  are  applicable  pri¬ 
marily.  to  those  procedures  and  rationales  which  use  analytical,  rather  than 


graphical  or  judgmental,  methods.  It  should  be  pointed  out  that  the  method 
for  achieving  simple  structure  by  means  of  a  binary  hypothesis  matrix,  such 
as  discussed  in  Chapter  17  for  the  special  case  of  the  multiple  group  method, 
might  be  regarded  as  a  special  case  of  a  direct  solution  from  the  correlation 
matrix.  Here,  however,  the  rationale  and  procedure  is  essentially  different 
from  that  of  the  methods  to  be  discussed  in  this  chapter. 

All  of  the  methods  discussed  in  this  chapter  are  based  on  the  analytical 
procedures  of  the  varimax  method,  and  constitute  applications  of  the  ration¬ 
ale  of  this  method  to  the  correlation  and  data  matrices.  The  direct  solu¬ 
tions  for  simple  structure  matrices  need  not  be  limited  to  the  varimax  type 
of  solution.  However,  as  in  the  previous  chapter,  they  will  be  so  restricted 
in  this  chapter  because  of  the  practical  difficulties  encountered  with  other 
alternatives  to  the  varimax  solutions  which  have  been  developed  and  experi¬ 
mented  with  so  far. 

19.1.3  Rank  Reduction  Solutions.  All  of  the  solutions  considered  in 
this  chapter  are  of  a  rank  reduction  type.  This  means  that  the  rank  of  the 
residual  matrix,  following  the  solution  of  any  factor  vector,  is  one  less 
than  that  of  the  previous  residual  matrix,  or,  in  the  case  of  a  factor  matrix, 
its  rank  is  equal  to  that  of  the  correlation  matrix  less  the  number  of  factors 
in  the  factor  matrix. 

19.1.4  Iterative  Solutions.  All  of  the  solutions  outlined  below  are 
of  an  iterative  type.  In  this  respect,  they  differ  from  the  direct  solution 
considered  for  the  multiple  group  method  in  Chapter  17,  in  which  a  binary 
hypothesis  was  used.  Here,  we  recall,  no  successive  approximations  were  re¬ 
quired,  except  for  the  orthonormal  transformation. 
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All  of  the  iterative  solutions  required  for  the  various  types  of  direct 
solutions  outlined  below  can  be  shown  to  be  special,  though  rather  compli¬ 
cated,  eigenvector  or  basic  structure  type  solutions  in  which  we  have  a  sym¬ 
metric  matrix,  some  of  whose  elements  are  functions  of  its  own  basic  ortho¬ 
normal  and  basic  diagonal  elements.  Therefore,  the  iterative  procedure  may 
be  somewhat  more  involved  than  in  the  case  of  a  straightf orward  basic  struc¬ 
ture  solution,  in  which  the  elements  of  the  symmetric  matrix  whose  basic 
structure  is  desired  are  constant  values.  As  a  matter  of  fact,  this  type 
of  basic  structure  solution  is  characteristic  not  only  of  the.  methods  .of ‘-this 
chapter,  but  also  of  those  of  the  previous  chapter,  in  which  the  solutions 
are  not  applied  directly  to  the  correlation  or  data  matrices  but  to  seme 
arbitrary  factor  matrix. 

19. 1.5  Utilization  of  Information.  One  of  the  distinguishing  charac¬ 
teristics  of  the  methods  outlined  in  this  chapter  is  that,  in  a  sense,  more 
of  the  information  in  the  correlation  or  data  matrix  is  utilized  than  in  the 
methods  of  Chapter  18.  Implicit  in  the  methods  of  both  Chapters  17  and  18 
is  the  assumption  that  the  factor  loading  matrix  accounts  for  all  of  the 
significant  or  systematic  nonrandom  information  inherent  in  the  correlation 
or  data  matrices.  This  assumption  may  not  be  valid  in  many  cases. 

In  the  methods  outlined  below,  all  of  the  information  in  the  data  or 
correlation  matrices  is  utilized  in  the  determination  of  the  simple  struc - 
ture  varimax  factor  matrices  or  vectors.  This  may  be  regarded  as  an  advant¬ 
age,  from  the  point  of  view  of  information  utilization;  or  it  may  be  regarded 
as  a  disadvantage,  if  one  takes  the  position  that  the  information  left  over 
after  the  major  product  moment  of  the  factor  loading  matrix  is  subtracted 


from  the  correlation  matrix  essentially  represents  error  or  random  variation. 
In  this  latter  case ,  one  may  argue  that  the  application  of  direct  varimax 
methods  to  the  correlation  or  the  data  matrices  may  he  spuriously  affected 
by  such  random  variation.  A  great  deal  more  theoretical  and  experimental 
work  needs  to  be  done  before  the  relative  validity  of  these  alternative 
points  of  view  can  be  established.  In  general,  the  criterion  of  invariance 
from  one  sample  of  entities  and  attributes  to  another  would  be  a  relevant 
consideration  here.  Presumably,  if  the  direct  methods  turn  out  to  yield 
more  consistent  results  from  one  sample  of  entities  to  another,  and  from 
one  sample  of  attributes  to  another,  their  superiority  would  be  definitely 
indicated. 

19.1.6  Results  Different  from  Transformation  Solutions.  It  must  be¬ 
come  obvious  that  one  cannot  expect  exactly  the  same  results  from  the  dir¬ 
ect  methods  outlined  below  as  from  the  transformation  solutions  indicated 
in  Chapters  17  and  18.  Even  for  the  direct  methods  of  Chapter  18,  in  which 
we  use  the  same  varimax  criteria  and  precisely  the  same  models,  one  cannot 
expect  to  get  exactly  the  same  results  as  when  the  criteria  are  applied  to 
the  correlation  or  data  matrices.  The  reason  for  this  is,  of  course,  that 
we  utilize  not  only  the  information  provided  by  seme  arbitrary  factor  load¬ 
ing  matrix,  but  also  information  inherent  in  the  data  or  correlation  matrix 
which  has  not  been  reflected  in  the  solution  for  the  arbitrary  factor  load¬ 
ing  matrix. 

Only  extensive  research  can  tell  which  of  the  methods  is  better  from 
the  point  of  view  of  factorial  invariance.  There  is,  of  course,  the  question 
of  which  factor  loadings  make  more  sense  from  the  point  of  view  of  the 


particular  discipline  concerned.  But  as  we  have  suggested  earlier,  the  de¬ 
termination  of  whether  or  not  the  results  make  sense  for  a  particular  dis¬ 
cipline  is  subjective.  Until  the  concept  is  more  objectively  defined  than 
it  has  been  in  the  past,  we  cannot  use  the  criterion  of  how  much  sense  the 
results  make  as  a  basis  for  comparison  of  any  of  the  methods  of  factor  analy¬ 
sis. 

19.2  Kinds  of  Methods 

For  convenience,  we  may  group  the  various  methods  into  four  classes. 

These  are  (l)  the  successive  factor  varimax  solution  from  the  correlation 
matrix,  (2)  the  simultaneous  factor  varimax  solution  from  the  correlation 
matrix,  (3)  the  successive  factor  varimax  solution  from  the  data  matrix,  and 
(4)  the  simultaneous  factor  varimax  solution  from  the  data  matrix.  Each  of 
these  classes  of  solutions  may,  in  the  conventional  manner,  maximize  the 
variance  of  the  squared  elements,  or,  more  generally,  the  variance  of  some 
other  even  power,  just  as  in  Chapter  18.  We  shall  consider  first  the  con¬ 
ventional  type  solutions,  and  then  the  general  type. 

19.2.1  Solutions  from  the  Correlation  Matrix.  As  indicated  in  the 
previous  section,  the  direct  solutions  may  proceed  either  by  operations  on 
the  correlation  matrix  or  by  operations  directly  on  the  data  matrix.  The 
methods  based  on  operations  on  the  correlation  matrix  may  again  be  of  two 
kinds,  analogous  to  the  two  types  of  varimax  rotations  for  arbitrary  factor 
matrices.  One  of  these  is  the  successive  factor  vector  method  which  obtains 
a  single  factor  vector  at  a  time.  The  other  is  the  simultaneous  factor  matrix 
method,  which  iterates  successively  to  the  entire  factor  loading  matrix  for 
the  particular  number  of  factors  hypothesized  to  be  significant  for  a  set 


of  variables. 


19.2.2  Solutions  from  the  Data  Matrix.  One  may  bypass  the  calculation 
of  the  correlation  matrix  and  operate  directly  upon  the  data  matrix.  Solu¬ 
tions  of  this  class  obviously  cannot  achieve  a  net  saving  in  computations 
over  the  number  required  for  the  correlation  matrix,  since,  as  one  may 
guess,  the  computations  for  the  direct  varimax  from  the  data  matrix  involve 
more  computations  than  solutions  based  on  the  correlation  matrix  itself. 

The  question  as  to  which  of  the  methods  is  most  economical  from  a  computer 
or  cost  point  of  view  depends  on  a  number  of  factors.  In  general,  if  the 
number  of  cases  is  not  vastly  greater  than  the  number  of  variables,  one  may 
save  sane  time  operating  directly  upon  the  data  matrix. 

In  any  case,  the  methods  outlined  here  assume  that  the  data  matrix  has 
been  scaled  so  that  the  variables  have  means  of  0  and  variances  of  unity. 

It  is  possible,  of  course,  to  work  out  computational  procedures  so  that  the 
data  matrix  need  not  first  be  processed  to  yield  a  standardized  metric.  Such 
a  computational  procedure  would  incorporate  the  vector  of  means  and  the  di¬ 
agonal  of  standard  deviations  or  variances.  These  methods,  like  those  in 
Section  19.2,1,  include  the  successive  factor  varimax  model  and  the  simul¬ 
taneous  factor  matrix  model,  but  here  they  are  applied  directly  to  the 
scaled  data  matrix. 

19.2.5  The  Successive  Factor  General  Varimax  Method.  The  convention¬ 
al  or  Kaiser  varimax  method,  as  we  know,  maximizes  the  variance  of  the  squared 
factor  loadings.  This  rationale,  we  recall,  may  be  generalized  so  that  the 
variance  of  any  even  power  function  of  the  factor  loading  variable  may  be 
maximized.  As  indicated  in  Chapter  18,  the  power  should  be  greater  than 
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unity.  Otherwise,  we  may  get  into  difficulty  with  reciprocals  of  very 
small  values.  The  successive  factor  general  varimax  procedures  may  again 
he  divided  into  two  models.  One  of  these  operates  on  the  correlation  matrix 
and  the  other  operates  directly  on  the  data  matrix. 

19.2.1  Simultaneous  Factor  General  Varimax.  Just  as  we  have  the  simul 
taneous  general  factor  varimax  method  operating  on  the  arbitrary  factor  load 
ing  matrix,  so  also  we  can  apply  this  model  directly  to  either  the  correla¬ 
tion  matrix  or  the  data  matrix.  In  this  chapter  we  shall  consider  its  ap¬ 
plication  to  both  the  correlation  matrix  and  the  data  matrix.  The  data 
matrix  is  assumed  to  be  scaled  so  as  to  give  means  of  0  and  variances  of 
unity. 

19-3  Successive  Varimax  Factor  from  Correlation  Matrix 

19.3.1  Characteristics  of  the  Method.  We  are  already  familiar  with 
some  of  the  characteristics  of  the  successive  factor  varimax  method  applied 
directly  to  the  correlation  matrix.  It  may  be  of  interest  to  compare  this 
method  with  others,  such  as  the  principal  axis  or  centroid,  with  respect  to 
the  amount  of  variance  accounted  for  by  the  successive  factors.  In  the 
latter  methods,  we  recall  that  the  amount  of  variance  accounted  for  tends 
to  decrease,  in  general,  with  the  successive  factors  calculated. 

We  cannot,  however,  assert  that  each  varimax  factor  calculated  from 
the  correlation  matrix  does  account  for  more  of  the  variance  than  the  sub¬ 
sequent  one.  We  cannot  even  guarantee  that  the  criterion  of  maximum  vari¬ 
ance  of  the  squared  factor  loadings  will  be  greater  for  a  given  factor 
vector  than  for  one  calculated  subsequently.  The  order  depends  very  largely 
on  the  characteristics  of  the  matrix,  and  also  on  what  is  used  as  a  first 
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approximation  for  any  particular  factor  loading  vector. 

A  further  reason  for  the  uncertainty  of  the  order  of  the  factors,  with 
respect  to  the  amount  of  variance  accounted  for  and  the  variance  of  their 
squared  factor  loadings,  is  the  fact  that  we  are  dealing  with  an  eigenvector 
or  basic  structure  problem  of  a  very  complicated  nature.  As  we  have  seen, 
the  model  involves  symmetric  matrices  whose  elements  are  functions  of  its 
eigenvectors  and  eigenvalues.  We  do  not  yet  have  available  an  adequate  mathe- 
matrical  substructure  for  a  satisfactory  understanding  of  what  determines  the 
order  in  which  the  factors  will  appear. 

19.3*2  Computational  Equations 
19.3*2a  Definition  of  Notation 

iR  is  the  ith  residual  correlation  matrix  where  is  the  correla¬ 
tion  matrix  itself. 

b  is  the  s  approximation  to  the  ith  varimax  factor  vector. 

7.  is  the  amount  of  variance  accounted  for  by  the  s  approximation 
to  the  ith  varimax  factor  vector. 


108 


871  '  sKl  .‘.l 

(19:3.2)- 

<s 

0  .  -  -  b 

s  .1  s  .i  sin 

(19.3.3) 

sW.i  -  iRs*.i  . 

(19.3.J0- 

s^i  "v/s^.i  s^.i 

(19.3.5) 

W  . 

_b  .  =  Sui 

8+1  3ai 

(19.3.6) 

r  =  R  -  I 
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(r 

u.o  -  p 1  -  h^r- 

(19.3.9) 

0  is  largest  element  in  U 

(19.3.10) 

lb.l  =  1R.L 

(19.3.11) 

19-3.3  Computational  Instructions.  We  begin  with  a  correlation  matrix. 
The  meaning  of  Eq.  (19.3-1)  is  as  follows.  If  i.  =  1,  we  have  simply  the  cor¬ 
relation  matrix.  The  right  hand  side  of  this  equation  will  then  be  ignored, 
because  the  i  -  1  would  be  0  and  have  no  meaning,  that  is,  it  would  not  be 
defined.  It  is  only  for  i  greater  than  1  that  Eq.  (19.3.1)  has  meaning. 

We  assume  now  that  we  have  a  first  approximation  to  a  factor  loading 
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vector  which  will  he  described  in  detail  in  Eqs,  ( 19.3.7)  through  (19.3.H). 
Eqs.  (19.3.2)  through  (19.3.6)  describe  the  successive  cycles  for  the  £  ap¬ 
proximation  to  the  ith  factor  loading  vector. 

Having  given  any  approximation  to  the  ith  factor  loading  vector,  its 
minor  product  moment  is  calculated  as  indicated  in  Eq.  (19.3.2) . 

Next  we  calculate  a  vector  as  indicated  in  Eq.  (19«3«3).  This  is  the 
£  vector  with  which  we  are  already  familiar.  It  is  obtained  by  cubing  the 
elements  of  the  b  vector,  and  subtracting  from  it  the  b  vector  multiplied 
by  the  scalar  in  Eq.  (19.3.2)  divided  by  n. 

The  next  step  is  indicated  by  Eq.  ( 19 . 3 • ^ ) ^  "which  again  is  of  course 
a  general  equation  for  the  £  approximation.  It  is  the  product  of  the  ith 
residual  correlation  matrix  postmultiplied  by  the  p  vector  calculated  in 
Eq.  (19.3.3). 

Next  we  calculate  the  scalar  indicated  by  Eq.  (19.3*5).  This  is  the 
square  root  of  the  minor  product  moment  of  the  vectors  of  Eqs.  (19.3.3) 
and  (19.3.4). 

The  s+1  approximation  to  the  varimax  factor  loading  vector  for  the  ith 
factor  is  calculated  in  Eq.  (19.3.6).  This  is  the  vector  of  Eq.  (19.3.4) 
divided  by  the  scalar  of  Eq.  (19.3*5). 

As  yet  there  appears  to  be  no  completely  satisfactory  method  for  choos¬ 
ing  a  first  approximation  to  any  particular  varimax  factor  loading  vector, 
including  the  first  one.  However,  the  following  method  is  recommended  and 
should  give  good  results  in  most  cases. 

Consider  the  correlation  matrix  with  0’s  in  the  diagonal,  as  indicated 
in  Eq.  (19.3.7). 
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First  we  square  each  element  of  the  matrix  r  of  Eq.  (19.3*7),  as  in¬ 
dicated  in  Eq.  (19.3.8). 

We  then  calculate  a  U  vector  as  indicated  in  Eq.  (l9»3»9).  This  is  ob¬ 
tained  by  calculating  a  vector  of  the  sums  of  the  rows  of  the  matrix  in  Eq. 
(19.3.8) •  From  this  is  subtracted  a  vector  of  the  squares  of  the  sums  of 
rows  of  the  matrix  in  Eq.  (19.3.7)  divided  by  n  -  1,  the  number  of  variables. 
The  U  vector  is  therefore  n  -  1  times  the  vector  of  the  variances  of  the 
columns  of  the  correlation  matrix  with  the  diagonal  elements  excluded. 

In  Eq.  (I9.3.IO),  we  find  the  largest  element  in  the  U  vector  calcu¬ 
lated  in  Eq.  (19.3*9)>  and  call  this  the  L  position. 

We  then  take  as  the  first  approximation  to  the  first  varimax  vector 
the  Lth  column  of  the  correlation  matrix,  as  indicated  in  Eq.  (19.3.H). 

Using  Eqs.  ( 19.3.2)  through  (19.3.6),  we  then  calculate  the  first  vari¬ 
max  factor  loading  vector  by  successive  iterations  until  the  a  scalar  of 
Eq.  (19.3.5)  stabilizes  to  some  prespecified  degree  of  accuracy. 

Then  we  calculate  the  first  residual  matrix  by  substituting  2  for  the 
i  subscript  in  Eq.  (19.3*1).  This  residual  matrix  is  simply  the  original 
correlation  matrix,  less  the  major  product  moment  of  the  final  approxima¬ 
tion  to  the  b  .  vector. 

We  now  require  a  first  approximation  to  the  second  varimax  factor 
vector.  We  apply  the  procedures  of  Eqs.  (19.3*7)  through  (19.3*11)  to  the 
residual  matrix. 

We  continue  with  Eqs.  (I9.5.I)  through  (19.3*6),  and  generalizations 
of  Eqs.  (19.3.7)  through  (19.3.11),  until  enough  factors  have  been  extracted. 
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19.3.4  Numerical  Example.  In  this  and  subsequent  numerical  examples 
in  this  chapter,  the  presentation  of  the  results  will  not  conform  to  that 
of  the  presentation  of  the  methods  themselves.  Rather  it  will  conform  to 
a  computer  program  sequence  which  is  more  efficient  for  performing  the  com¬ 
putations  included  in  all  of  the  methods  of  this  chapter.  Each  numerical 
example  section  will  include  both  the  conventional  case  in  which  the  vari¬ 
ance  of  squared  elements  of  the  varimax  factor  vectors  are  maximized,  and 
a  special  example  of  the  general  case  for  the  ^  power. 

The  same  correlation  matrix  used  in  previous  chapters  will  be  used  for 
all  of  those  methods  beginning  with  the  correlation  matrix.  This  is  repeated 
for  convenient  reference  in  Table  19.3.1. 

The  same  data  matrix  will  be  used  for  all  methods  operating  directly 
on  the  data  matrix.  This  matrix  yields  the  correlation  matrix  of  Table 
19,3.1,  so  that  the  results  obtained  from  the  data  matrix  may  be  compared 
with  those  obtained  from  the  correlation  matrix. 

Table  19.3*2  gives  the  data  matrix. 

Table  19.3*3  gives  the  varimax  factor  matrix  for  three  factors  obtained 
by  the  successive  factor  matrix  method  directly  from  the  correlation  matrix, 
for  the  conventional  case  in  which  the  variances  of  the  elements  of  the 
squared  factor  loading  vectors  are  maximized. 

Table  19.3*4  gives  the  varimax  factor  loading  matrix  for  the  successive 
factor  general  matrix,  where  the  power,  rather  than  the  square  of  the 
elements,  is  used. 
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Table  19.3*1  -  Correlation  Matrix 


1.000 

0.829 

O.768 

0.108 

0.033 

0.108 

0.298 

0.309 

0.351 

0.829 

1.000 

0.775 

0.115 

0.061 

0.125 

O.323 

0.347 

0.369 

0.768 

0.775 

1.000 

0.272 

0.205 

0.238 

0.296 

0.271 

0.385 

0.108 

0.115 

0.272 

1.000 

O.636 

0.626 

0.249 

0.183 

0.369 

0.033 

0.061 

0.205 

0.636 

1.000 

O.709 

0.138 

0.091 

0.254 

0.108 

0.125 

0.238 

0.626 

0.709 

1.000 

0.190 

0.103 

0.291 

0.298 

0.323 

0.296 

0.249 

0.138 

0.190 

1.000 

0.654 

O.527 

0.309 

0.347 

0.271 

0.183 

0.091 

0.103 

0.654 

1.000 

0.541 

0.351 

0.369 

0.385 

0.369 

0.254 

0.291 

0.527 

0.541 

1.000 

Table  19.3.2 

1  -  Normalized  Deviation 

Data  Matrix 

0.128 

0.181 

0.421 

0.506 

0.857 

0.746 

0.280 

0.178 

0.246 

0.76k 

0.740 

0.563 

-0.387 

-0.293 

-0.202 

0.261 

0.281 

0.043 

-0.030 

-0.046 

0.014 

0.147 

-0.109 

-0.135 

0.640 

0.682 

0.661 

-0.280 

-0.351 

-0.326 

-0.023 

-0.109 

-0.186 

0.083 

0.091 

-0.654 

-0.336 

-O.306 

-0.429 

-0.542 

-0.006 

-0.153 

-0.428 

0.056 

-0.124 

-0.276 

-0.324 

-0.271 

-0.370 

-0.225 

0.035 

0.129 

-0.446 

-0.124 

0.057 

-0.070 

-0.016 

0.006 

0.152 

-0.441 

-0.166 

-0.354 

-0.033 

-0.010 

-0.140 

0.326 

-0.004 

-0.258 

-0.091 

-0.410 

-0.200 

-0.101 

-0.303 

0.227 

-0.014 

0.029 

-0.102 

-0.125 

-0.086 

-0.106 

-0.074 

0.086 

0.057 

-0.003 

0.161 

0.073 

0.134 

-0.082 

-0.067 

-0.009 

0.164 

0.106 

-0.124 

0.234 

-0.002 

0.227 

-0.072 

0.025 

0.156 

0.036 

-0.074 

-0.142 

0.242 

0.023 

0.192 

-0.148 

-0.119 

0.011 

Table  19.3.3  -  Successive  Factor  Varimax  Matrix  from  Correlation  Matrix 
f or  k  =  2 


0.937 

0.003 

0.092 

0.933 

0.023 

0.127 

0.887 

0.175 

0.063 

0.098 

0.782  ' 

0.130 

0.042 

0.921 

0.026 

0.100 

0.894 

0.044 

0.234 

0.129 

0.837 

0.243 

0.058 

0.903 

0.300 

0.247 

0.527 
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Table  19.3.4.  -  Successive  Factor  General  Varimax  Matrix  from  Correlation 
Matrix  for  k  =  ^ 

0.9.4?  -0.040  0.018 
0.946  -0.013  0.0.57 
0.880  0.140  -0.012 
0.143  0.663  0.099 
0.078  0.991  0.000 
0.143  0.769  0.008 
0.302  0.123  0.596 
0.311  O.067  0.948 
0.364  0.235  0.430 
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19 .4  Simultaneous  Varimax  Matrix  from  the  Correlation  Matrix 


19.4.1  Characteristics  of  the  Method.  This  method  is  like  the  preced¬ 
ing  one  in  that  we  operate  directly  on  the  correlation  matrix  rather  than  on 
the  arbitrary  factor  matrix.  Here,  however,  we  iterate  simultaneously  to 
all  of  the  factor  loading  vectors  which  we  wish  to  solve  for.  It  differs 
also  from  the  previous  method  in  that  we  have  a  less  objective  way  for  getting 
a  first  approximation  to  the  factor  loading  vectors  than  we  did  in  that  method. 
It  is  also  essentially  different  computationally  from  the  previous  method  in 

that  each  iteration  involves  the  solution  for  the  basic  structure  of  a  sym- 

o 

metric  matrix.  The  Q  orthonormal  and  A_  basic  diagonal  of  this  matrix  are 
required  in  the  successive  approximations  to  the  factor  loading  vectors.  In 
this  respect  the  method  is  analogous  to  the  simultaneous  method  of  Chapter 
18. 

19.4.2  Computational  Equations 
19.4.2a  Definition  of  Notation 

R  is  the  correlation  matrix. 

flb  is  the  s  approximation  to  the  varimax  factor  matrix. 

b^  is  a  matrix  whose  elements  are  the  cubes  of  those  in  b, 
s  s 

D  .  /  .  is  a  diagonal  matrix  whose  elements  are  the  diagonals  of 

8°  gD 

the  minor  product  moment  of  b. 

s 

Q  is  a  basic  orthonormal, 
s 

o 

a A '  is  a  basic  diagonal. 
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19.4.2b  The  Equations 

/ D  b'  b 

p  =  b(3)  -  b  ~S . »■ 

sH  s  s  n 


M  =  R  p 

3  SK 


p1  M 
sK  s 


s+1 


b  =  «SM  sQ)  sQ/ 


(19.4.1) 

(19.4.2) 
(19.4.5) 

(19.4.4) 

(19.4.5) 


19.4.3  Computational  Instructions.  In  this  method  we  start  with  some 
arbitrary  approximation  to  the  varimax  factor  loading  matrix.  This  may  be 
simply  the  first  m  columns  of  the  correlation  matrix,  where  m  is  the  number 
of  factors.  In  any  case,  one  must  make  an  assumption  as  to  the  number  of  sig¬ 
nificant  factors  in  the  data  matrix.  It  is  better  to  overestimate  than  to 
underestimate  the  number,  as  some  of  the  factors  can  later  be  rejected  from 
the  final  stabilized  varimax  factor  loading  matrix  if  they  do  not  seem  inter¬ 
pretable  or  of  sufficient  importance. 

If  one  has  an  hypothesis  as  to  which  of  the  variables  represents  which 
factor,  he  may  select  a  variable  to  represent  each  factor.  Then  the  columns 
of  the  correlation  matrix  corresponding  to  these  variables  will  constitute 
the  vectors  of  the  first  approximation  to  the  varimax  factor  matrix. 

We  begin  then  with  Eq.  (19.4.1)  in  which  the  subscript  £  is  taken  as  1. 
This  equation  is  similar  to  those  we  are  already  familiar  with.  The  right 
hand  side  has  for  the  first  terra  a  matrix  whose  elements  are  the  cubes  of 
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the  elements  of  the  corresponding  approximation  to  the  factor  loading  matrix. 
We  subtract  from  this  the  factor  loading  matrix  itself,  after  scaling  it  by 
a  diagonal  matrix.  This  diagonal  matrix  is  made  up  of  the  diagonals  of  the 
minor  product  moment  of  the  current  approximation  to  the  factor  loading 
matrix,  and  then  divided  by  n,  the  number  of  variables. 

Next  we  calculate  the  product  indicated  in  E^.  (19.4.2).  This  is  the 
correlation  matrix  postmultiplied  by  the  matrix  calculated  in  Eq.  (19.4.I). 

We  then  calculate  the  minor  product  of  the  matrices  calculated  in  Eqs. 
(19.4.1)  and  (19.4.2).  This  is  the  matrix  G  indicated  in  Eq.  (19.4.3). 

Next  we  calculate  the  basic  structure  factors  of  the  matrix  G.  This  is 
indicated  by  Eq.  (19.4.4). 

Finally,  for  each  approximation,  we  calculate  the  product  of  the  four 
factors  as  indicated  in  Eq.  (19.4.5).  This  is  the  product  of  the  matrix  of 
Eq.  (19.4.2)  postmultiplied  first  by  the  right  orthonormal  of  the  matrix  in 
Eq.  (19.4.4),  then  by  the  inverse  of  the  square  root  of  the  basic  diagonal 
of  the  matrix  in  Eq.  (19.4,4),  and  finally  by  the  left  orthonormal  of  the 
matrix  in  Eq.  (19.4.4),  which  is,  of  course,  the  transpose  of  the  right 
orthonormal.  This  is  true  because  G  is  a  product  moment  matrix.  We  now 
have  the  s+1  approximation  to  the  matrix  of  varimax  factor  loadings.  These 
iterations  continue  until  the  approximation  is  sufficiently  close. 

A  good  criterion  of  convergence  is  the  trace  of  the  G  matrix  given  by 
Eq.  (19.4.3).  Another  criterion  may  be  the  trace  of  the  minor  product  moment 
of  the  current  approximation  to  the  factor  loading  matrix.  This  would  be  the 
diagonal  matrix  in  the  right  term  of  the  right  side  of  Eq.  (19.4.1).  This 
trace  is  simply  the  total  amount  of  variance  accounted  for  by  any  particular 


s  approximation  to  the  factor  loading  matrix.  As  indicated  earlier  in  this 
chapter,  any  approximation  is  a  rank  reduction  solution,  and  therefore  the 
larger  this  trace  the  greater  the  amount  of  variance  accounted  for. 

19.4.4  Numerical  Example.  This  example  begins  with  the  same  correla¬ 
tion  matrix  given  in  Table  19.3.I. 

Table  19.4.1  gives  the  first  three  varimax  factor  vectors  for  the  simul¬ 
taneous  factor  method  applied  to  the  correlation  matrix  for  the  convention¬ 
al  case  of  the  squared  elements. 

Table  19.4.2  gives  the  first  three  varimax  factor  vectors  for  the  simul¬ 
taneous  factor  general  method  applied  to  the  correlation  matrix  for  the  case 
of  the  ~  power  of  the  varimax  factor  elements.  The  results  are  not  markedly 
different  from  those  in  Table  19.4.1. 
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Table  19*4.1  -  Simultaneous 
Matrix  for  k 


Factor  Varim'ax  Matrix  from  Correlation 
=  2 


0*929 

0.007 

•0.153 

.0*922 

0.025 

0.187 

0.882 

0.179 

0.125 

.0*085 

0.781 

0.153 

.0*053 

0.917 

0.048 

0.090 

O.896 

0-.072 

•0.177 

0.112 

0.884 

0.186 

0.040 

O.891 

0.266 

0.238 

0.551 

Table  19*4.2  -  Simultaneous  Factor  General  Varimax  Matrix  from  Correla¬ 
tion  Matrix  for  k  =  ^ 

-  3 


0.944 

0.018 

0.121 

0.935 

0.024 

0.148 

0.857 

0.194 

0.112 

0.080 

0.989 

0.116 

0.037 

0.670 

0.049 

0.100 

0.645 

0.092 

0.189 

0.123 

0.974 

0.242 

0.088 

O.633 

0.306 

0.292 

0.442 

I 
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19*5  Successive  Factor  Vector  from  Data  Matrix 

19.5.1  Characteristics  of  the  Method.  This  method  differs  essential¬ 
ly  from  the  preceding  methods  in  that  the  calculation  of  the  correlation 
matrix  is  not  required.  The  computations  proceed  directly  upon  the  data 
matrix.  It  is  assumed  that  they  have  been  previously  scaled.  This  assump¬ 
tion  is  not  imperative,  however,  since  the  computations  could  be  modified 
to  operate  on  a  raw  data  matrix.  The  method  differs  also  from  the  succes¬ 
sive  factor  matrix  method  of  Section  19.3  in  that  it  is  difficult  to  select 
a  first  approximation  to  a  factor  loading  vector  by  objective  means.  Per¬ 
haps  the  simplest  way  to  get  the  first  approximation  to  the  first  factor 
loading  vector  is  to  assume  that  the  first  variable  is  not  an  extremely 
poor  representation  of  the  first  varimax  factor. 

19.5.2  Computational  Equations 

19.5.2a  Definition  of  Notation 


^Z  is  the  ith  residual  data  matrix  where  ^Z  is  the  scaled  data 
matrix  with  0  means  and  unit  variances, 

(5) 

7. ,  b  ,  b'J',  and  a,  are  the  same  as  in  Section  19.3.2a. 

5  X  j3*i.  S  1 

Y  .  is  the  s  approximation  to  the  ith  varimax  factor  score  vector. 

S  *1  *"  " 


19.5.2b  The  Equations 


iz  -  i-iz  -  Y,(i-i)  b!(i-i) 


(19.5.D 


.b  .  =  ,Z'  .Z  T 

1  .i  i  i  .If. 


•s» 


(19.5.2) 
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s7i  =  sb,.i  sb.i 

(19.5.3) 

p  =  t>(2)  _  £i 

Sp.i  s  ,i  s  ,i  n 

(19-5.4) 

sU.i  =  i^.i 

(19.5.5) 

sai  =,/ZT7I 

(19.5.6) 

Y  8U.i 

3  -1  =  sai 

(19-5.7) 

s+lb.i  =  iZV.i 

(19.5.8) 

u.l  =  b.l 

(19.5.9) 

U,  ,  =  smallest  U  , 

(19.5.10) 

u.i  -  u.i-i  +  b.i-i 

(19.5.11) 

U-  ,  =  smallest  U  . 

•Wj  1  *1 

(19.5.12) 

19«  5*3  Computational  Instructions,  We  assume  that  a  normalized  data 
matrix  is  available.  Ordinarily,  one  would  not  normalize  a  complete  data 
matrix  with  a  large  number  of  variables  if  the  computations  are  done  with  a 
desk  computer.  It  is  assumed,  however,  that  for  this  particular  model  a 
high  speed  computer  is  available.  It  is  also  assumed  that  a  preliminary 
computer  program  is  available  for  transforming  the  raw  data  matrix  to  one 
whose  means  are  0  and  whose  variances  are  unity. 


4, 


this  is  the  standard- 


We  begin  by  considering  Eq.  (19.5*1).  If  i  =  1, 
ized  data  matrix  and  we  bypass  this  equation  to  get  a  first  approximation  to 
the  first  factor  loading  vector. 

This  is  indicated  in  Eq.  (19.5.2).  On  the  basis  of  some  rationale  or 
hypothesis,  we  may  select  seme  particular  variable  as  a  satisfactory  approxi¬ 
mation  to  one  of  the  factors.  If  no  satisfactory  rationale  is  available,  we 
may  arbitrarily  begin  with  the  first  variable.  It  is  seen,  therefore,  that 
the  vector  given  by  Eq.  (19.5*2)  is  the  correlation  of  the  selected  variable 
L  with  all  of  the  variables,  including  itself. 

Eqs.  (19.5.3)  through  (19.5.8)  give  the  successive  cycles  required  for 
a  particular  approximation  to  the  factor  loading  vector  b  We  shall  dis¬ 
cuss  this  set  of  computations  before  indicating  generally  how  we  get  the 
first  approximation  for  any  particular  factor  vector  following  the  first. 

Eq.  (19.5.3)  shows  the  minor  product  moment  of  the  current  approxima¬ 
tion  to  the  ith  factor  vector.  This  scalar,  7_,  indicates  the  amount  of 
variance  accounted  for  by  the  £  approximation  to  the  ith  factor  vector. 

The  0  vector  is  given  in  Eq.  (19.5*^).  This  is  obtained  by  construct¬ 
ing  first  a  vector  of  the  cubes  of  the  elements  in  the  current  approximation 
to  the  ith  factor  vector,  and  subtracting  from  it  the  current  approximation 
multiplied  by  the  scalar  of  Eq.  (19.5.3)  divided  by  n. 

Next  we  compute  the  product  indicated  in  Eq.  (19*5*5)>  which  is  the 
ith  residual  data  matrix  postmultiplied  by  the  vector  of  Eq.  (19.5**0* 

Then  we  calculate  the  scalar  indicated  by  Eq,  (19.5*6)  which  is  the 
square  root  of  the  minor  product  moment  of  the  vector  calculated  in  Eq. 
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We  next  calculate  the  current  approximation  to  the  factor  score  vector 

Y  .  as  indicated  in  Eg.  (19.5.7).  This  is  the  vector  calculated  in  Eg. 

»1 

(19.5*5)  divided  hy  the  scalar  calculated  in  Eg.  (19.5.6). 

Finally,  we  calculate  the  s+1  approximation  to  the  b  .  vector  by  means 
of  Eg.  (19.5.8).  This  is  the  transpose  of  the  ith  residual  data  matrix  post- 
multiplied  by  the  factor  score  vector  of  Eg.  ( 19.5.7). 

We  then  begin  again  with  Eg.  ( 19. 5*3)  and  repeat  the  cycle.  We  contin¬ 
ue  this  set  of  iterations  until  either  the  7  scalar  of  Eg.  (19.5*3)  or  the 
a  scalar  of  Eg.  (19.5*6)  is  stabilized  to  some  specified  degree. 

Then  we  return  to  Eg.  (19.5.1)  to  calculate  a  new  residual  matrix,  which 
is  obtained  by  subtracting  the  major  product  of  the  stabilized  Y  and  b  vectors 
of  Eqs.  (19.5.7)  and  (19.5.8),  respectively,  from  the  current  residual  matrix. 

To  get  the  first  approximation  to  the  second  varimax  factor  vector,  we 
consider  Eg.  (19.5.9).  Here  we  simply  equate  the  U  .  vector  to  the  first 

ti 

stabilized  factor  loading  vector  b  , . 

We  then  find  the  algebraically  smallest  element  in  the  vector  of  Eq. 
(19.5.9),  as  indicated  in  Eq.  (19.5.10). 

Next  we  get  the  first  approximation  to  the  second  factor  vector  by 
letting  i  =  2  in  Eg.  (19.5.2). 

To  get  the  first  approximation  to  the  ith  factor  loading  vector  we  con¬ 
sider  Eg.  (19.5.11).  To  get  U  we  add  U  ±_1  to  the  stabilized  b  ^  vari¬ 
max  vector. 

Eg.  (19.5.12)  indicates  the  algebraically  smallest  element  in  a  vector 
of  Eg.  (19.5.H).  This  we  designate  as  in  the  position. 


Then  we  return  to  Eq..  (19.5.2)  to  get  the  first  approximation  to  the 
ith  varimax  factor  loading  vector.  This  is  the  transpose  of  the  ith  resi¬ 
dual  data  matrix  poataultiplied  by  the  column  of  this  residual  matrix. 

19. 5. ^  Numerical  Example.  This  numerical  example  begins  with  the 
data  matrix  given  in  Table  19.3.2. 

Table  19.5.1  shows  the  first  three  varimax  factor  vectors  obtained  by 
the  successive  factor  method  directly  from  the  data  matrix  for  the  case  of 
k  =  2. 

Table  19.5.2  gives  the  varimax  factor  matrix  obtained  by  the  succes¬ 
sive  factor  general  method  from  the  data  matrix  for  the  case  of  k  =  —  . 
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Table  19. 5.1 

-  Successive  Factor  Varimax  Matrix  from  Data  Matrix  for 
K  »  2 

0.934 

O.091 

-0.010 

0.934 

0.127 

0.016 

O.89I 

0.076 

0.169 

0.093 

O.I65 

0.726 

0.044 

0.077 

0.932 

0.096 

0.093 

0.839 

0.236 

0.881 

0.08l 

0.240 

0.871 

0.012 

0.300 

0.541 

0.212 

Table  19.5*2  -  Successive  Factor  General  Varimax  Matrix  from  Data  matrix 


0.943 

-0.226 

-0.052 

0.947 

0.320 

-0.006 

0.884 

-0.156 

0.136 

O.136 

-0.050 

0.606 

0.079 

-0.019 

0.991 

4 

O.I36 

-0.016 

0.700 

0.304 

0.072 

0.124 

0.307 

0.107 

0.069 

O.363 

0.048 

0.225 

i 
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19.6  Simultaneous  Factor  Matrix  from  Data  Matrix 

19*6.1  Characteristics  of  the  Method.  The  characteristics  of  this 
method  have  already  been  fairly  well  covered  in  the  previous  sections.  Ex¬ 
cept  for  decimal  accuracy,  it  should  give  essentially  the  same  results  as 
the  method  discussed  in  Section  19.4.  The  calculation  of  the  correlation 
matrix  as  such  is  bypassed,  and  the  multiplications  implied  by  the  correla¬ 
tion  matrix,  that  is,  the  minor  product  moment  of  the  data  matrix,  is  ac¬ 
complished  at  each  iteration  by  two  successive  multiplications  of  a  matrix 
by  a  vector. 

The  method  avoids  the  accumulation  of  decimal  error  resulting  from  the 
calculation  of  residual  matrices,  such  as  in  Sections  19. 3  and  19*5.  How¬ 
ever,  as  in  Section  19.4,  for  each  approximation  one  must  calculate  the 
basic  orthonormal  and  basic  diagonal  of  a  Gramian  matrix  whose  order  is 
equal  to  the  number  of  factors.  Again,  this  is  not  a  formidable  task  for 
high  speed  computers,  since  a  number  of  computer  programs  are  already  avail¬ 
able  for  computing  all  of  the  latent  roots  and  vectors  of  the  Gramian  matrix, 
including  the  programs  in  the  appendix  for  Chapter  9. 

19.6.2  Computational  Equations 
19.6.2a  Definition  of  Notations 

Z  is  the  data  matrix  with  means  of  0  and  variances  of  unity. 

(l)  2 

b,  b'  ,  Q,  and  A  are  the  same  as  in  Section  19-5 -2b. 

3  S _  S  8 

sY  is  the  £  approximation  to  the  varimax  factor  score  matrix. 
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19.6.2b  The  Equations 


lb  ■  Z'Z(m) 

(19.6.1) 

p  =  -  b  3  s 

s  s  s  n 

(19.6.2) 

s°  • 

(19.6.3) 

(19.6.4) 

s«ai2s«'  =  5C! 

(19.6.5) 

s0"*  -  .«  /.« 

(19.6.6) 

Y  =  U  G"^ 

8  S  S 

(19.6.7) 

.,b  =  Z  /  Y 
s+1  s 

(19.6.8) 

19.6.3  Computational  Instructions,  In  this  model  we  begin  with  a 
standardized  data  matrix. 

We  must  choose  seme  sort  of  approximation  to  the  first  varimax  factor 
loading  matrix.  If  we  have  some  hypothesis  as  to  a  single  variable  which 
best  measures  each  of  the  factors,  we  can  use  these  variables  to  begin  the 
computations.  In  any  case,  whether  we  have  a  rational  procedure,  or  whether 
we  select  the  first  m  variables  where  m  is  the  number  of  factors  we  expect 
to  solve  for,  we  begin  with  Eq.  (19,6.1).  The  right  side  of  this  equation 
iB  the  transpose  of  the  data  matrix  postmultiplied  by  a  submatrix  made  up 
of  m  vectors  out  of  Z.  These  m  vectors  may  be  rationally  or  arbitrarily 
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selected.  Actually,  then,  this  first  approximation  to  the  b  matrix  is  simply 
a  matrix  of  the  correlations  of  the  m  variables  with  all  the  variables,  in¬ 
cluding  the  correlations  among  themselves. 

The  general  equations  for  the  computations  are  then  indicated  by  Eqs. 
(19.6.2)  through  (19.6.7). 

Eq.  (19.6.2)  gives  the  computation  for  the  first  approximation  to  the 
£  matrix,  just  as  in  Section  19.4.2.  The  first  term  on  the  right  of  this 
equation  is  a  matrix  whose  elements  are  the  cubes  of  the  corresponding  ele¬ 
ments  of  the  current  approximation  to  the  b  or  varimax  factor  loading  matrix. 
From  this  is  subtracted  the  current  approximation  to  the  b  matrix,  scaled 
by  a  diagonal  matrix  on  the  right.  This  diagonal  matrix  is  made  up  of  the 
diagonal  elements  of  the  minor  product  moment  of  the  current  approximation 
to  the  b  matrix,  divided  by  n,  the  number  of  variables. 

The  next  step  is  given  by  Eq.  (19.6.J).  As  indicated  on  the  right  hand 
side  of  this  equation,  the  data  matrix  Z  is  postmultiplied  by  the  £  matrix 
given  in  Eq.  (19.6.2). 

The  next  step  is  given  in  Eq.  (19.6.4).  The  matrix  G  is  the  minor  pro¬ 
duct  moment  of  the  matrix  calculated  In  Eq.  (19.6.3). 

We  then  calculate  the  basic  structure  factors  of  the  matrix  G  given  in 
Eq.  (19.6.4),  as  indicated  by  the  left  hand  side  of  Eq.  (19.6.5).  The  com¬ 
puter  programs  given  for  Chapter  9  Tor  finding  basic  structure  factors,  or 

eigenvalues  and  eigenvectors,  of  symmetric  matrices  are  applicable  here. 

1 

Next  we  calculate  the  G~?  matrix  of  Eq.  (19.6.6).  This  is  the  triple 
product  involving  the  factors  obtained  from  Eq,  (19.6.5). 
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We  then  calculate  the  current  approximation  to  the  varimax  factor 
score  matrix,  as  indicated  in  Eq.  (19.6.7)*  This  is  the  matrix  of  Eq. 
(19.6.3)  postmultiplied  by  the  matrix  of  Eq.  (19.6.6). 

Finally,  we  calculate  the  next  approximation  to  the  b  or  varimax  factor 
loading  matrix,  as  indicated  in  Eq.  (19.6.8).  This  is  simply  the  transpose 
of  the  data  matrix  Z  postmultiplied  by  the  factor  score  matrix  given  in  Eq. 
(19.6.7). 

These  computations  continue  until  either  the  trace  of  the  minor  product 
moment  of  the  current  factor  loading  approximation  matrix,  or  the  trace  of 
the  G  matrix  in  Eq.  (19.6.4),  reaches  some  specified  degree  of  stabiliza¬ 
tion. 

19.6.4  Numerical  Example,  This  numerical  example  begins  with  the  data 
matrix  given  in  Table  19.3.2. 

Table  I9.6.I  gives  the  first  three  varimax  factor  vectors  obtained  by 
the  simultaneous  factor  matrix  method  directly  from  the  data  matrix  for  the 
case  of  k  =  2. 

Table  19.6.2  shows  the  varimax  factor  matrix  obtained  by  the  simul¬ 
taneous  factor  general  matrix  method  directly  from  the  data  matrix  for  the 

case  of  k  =  ^  . 

-  3 
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Table  19.6.1  -  Simultaneous  Factor  Varimax  Matrix  from  Data  Matrix 
for  k  =  2 


0.952 

0.000 

0.161 

0.952 

0.026 

0.192 

0.895 

0.180 

0.121 

0.082 

0.755 

0.145 

0.040 

0.961 

0.051 

0.089 

0.874 

0.064 

0.181 

0.091 

0.901 

0.185 

0.021 

0.899 

0.269 

0.227 

0.562 

Table  19.6.2  -  Simultaneous  Factor  General  Varimax  Matrix  from  Data 


Matrix  for  k  = 

10 

0.945 

0.018 

0.136 

3 

0.948 

0.054 

0.147 

0.874 

0.224 

0.066 

0.085 

1.056 

0.087 

0.046 

0.694 

0.012 

0.102 

0.630 

0.093 

0.194 

0.118 

0.972 

0.241 

0.081 

0.641 

0.312 

0.302 

0.446 

ljo 


19*7  Successive  Factor  General  Varimax 

As  we  recall,  the  general  varimax  method  is  similar  to  the  Kaiser  vari¬ 
max  method,  except  that  it  is  based  on  sane  even  fractional  power,  greater 
than  unity,  of  the  elements  whose  variance  is  maximized,  rather  than  on  the 
squares  of  these  elements. 

19.7*1  Computational  Equations 
19.7.1a  Definition  of  Notation 


m^  is  an  integer  not  less  than  1. 
m2  is  an  integer  not  greater  than  m^. 

b  .  is  the  s  approximation  to  the  ith  varimax  factor  vector. 
t(2k-l)  ^(k-l)  are  vectors  whose  elements  are,  re- 

O  >1  8  >1  S  el 

spectively,  the  2k -1,  k-1,  and  k  powers  of  those  in  b  , . 

"*  *“  ™  Bel 


19.7.1b  The  Equations 


2  m. 


k  = 


2  mg  -  1 


e  ,  .  b'f -1)  . 

s  .i  s  ,i 


1'  b^ 
b(k-!)  1  sb.i 

s  ,i  n 


(19.7.1) 

(19.7.2) 


19.7.2  Computational  Instructions.  The  procedures  here  are  precisely 
the  same  as  in  Sections  19.3  and  19.5,  respectively,  except  that  the  £  vectors 
are  calculated  in  a  different  manner,  since  the  function  whose  variance  we 
want  to  maximize  is  more  general  than  that  of  the  square  of  the  factor  load¬ 
ing. 
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We  begin  with  Eq.  (19.7.I).  Here  we  have,  instead  of  the  second  power, 
the  kth  power  of  the  elements  of  the  factor  loading  vectors  whose  variance 
we  wish  to  maximize.  This  is  expressed  as  the  ratio  of  twice  the  sum  of  a 
positive  integer,  divided  by  twice  some  other  positive  integer  less  1,  The 
positive  integer  in  the  denominator  of  this  equation  cannot  be  greater  than 
that  in  the  numerator. 

To  define  the  £  approximation  to  the  j3  vector  corresponding  to  the  ith 
varimax  factor  loading  vector,  we  use  Eq.  (19.7.2).  This  is  the  same  as  Eq. 
(I8.5.I)  of  Chapter  18.  As  indicated  in  that  chapter,  either  we  will  re¬ 
quire  tables  of  logs  and  exponentials  to  calculate  the  powers  of  the  elements 
of  b  indicated  on  the  right  hand  side  of  Eq.  (19.7.2),  or  library  functions 
.  for  the  computer  program  must  be  available. 

19.8  Simultaneous  General  Varimax 

The  simultaneous  general  varimax  procedure  is  similar  to  procedures  de¬ 
scribed  in  Sections  19.4  and  19.6,  except  for  the  power  of  the  elements  in 
the  simple  structure  matrices  whose  variances  are  to  be  maximized. 

19.8.1  Computational  Equations 

19.8.1a  Definition  of  Notation 

m^  and  m2  are  the  same  as  in  Section  19.7- 2a. 

gb  is  the  £  approximation  to  the  varimax  factor  matrix. 

b^^  b^k\  b^k-1^  are  matrices  whose  elements  are,  respec- 
S  S  '  s _ 

“  tively,  the  2k -1,  k,  and  k-1  powers  of  the  corresponding  ele¬ 

ments  in  b. 

1  s 
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19.8.1b  The  Equations 


k 


2  m. 


2  m2  -  1 


p  =  b^-1) 

s  s 


b(k-l) 


Dl'  b(k) 
s 

n 


(19.8.1) 

(19.8.2) 


19.8.2  Computational  Instructions.  The  computational  instructions  are 
the  -same  as  for  the  method  using  the  correlation  matrix,  given  in  Section 
19.^,  and  the  method  using  the  data  matrix,  given  in  Section  19.6,  except 
for  the  calculation  of  the  p  matrices. 

We  shall  first  consider  the  general  case  for  both  the  correlation  and 
the  data  matrices.  Again,  as  in  Section  19.7,  we  begin  with  Eq.  (19.8.1) 
which  gives  the  value  of  k  as  the  power  of  the  elements  of  the  simple  struct 
ture  factor  loading  vector  whose  variance  we  wish  to  maximize.  The  restric¬ 
tions  on  and  m^  on  the  right  hand  side  of  this  equation  are  the  same  as 
the  previous  ones. 

The  general  equation  for  the  (3  matrix  is  now  given  by  Eq.  (19.8.2), 
where  the  exponents  in  parentheses  for  the  b  matrices  indicate  that  the  cor¬ 
responding  elements  of  the  current  approximation  to  the  b  matrix  have  been 
raised  to  the  indicated  power.  The  diagonal  matrix  on  the  right  of  the  right 
hand  term  of  the  right  side  of  Eq.  (19.8.2)  is  obtained  as  follows.  We  sum 
the  columns  of  the  matrix  whose  elements  are  the  kth  power  of  the  elements 
in  the  b  matrix,  and  use  the  elements  of  this  vector  in  the  diagonal.  This 
diagonal  is  then  divided  by  n,  and  the  b_  matrix  with  elements  raised  to  the 


k  -  1  power  is  scaled  accordingly, 
19.9  Mathematical  Proofs 


19.9*1  Successive  Faetor  Matrix  from  Correlation  Matrix. 

Fran  Section  18.3  we  have,  as  the  iterative  solution  for  the  varimax 
factor  vector  h  . , 


Ai  - 

•>* 

s  ,i  s  ,i  n 

(19.9.1) 

sU.i  " 

ia'  .P.i  • 

(19.9.2) 

Sai  = 

Mi  su.i 

(19.9.5) 

sH.i  - 

su.i 

A 

(19.9A) 

s+lb.i 

=  ia  sH.i 

(19.9.5) 

where 

ia  = 

i-la  "  b.(i-l)  H!(i-1) 

(19.9.6) 

From  Eq. 

(19.9.2) 

ia  i&/ 

sP.i  =  ia  sU.l  . 

(19.9.7) 

Fran  Eqs, 

,  (19.9.5)  and  (19.9.6) 

ia  la' 

=  (i-l)a  (i-l)a'  "  b.(i-l)  b'.(i-l) 

(19.9.8) 

From  Eqs.  (19.9*1*),  (19.9.5),  and  (19.9*7) 
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Prom  Eq.  (19.9*3) 


A  •//'.  1 1“  iv  .p.i 


(19.9.10) 


We  now  let 


iB  -  iaia' 


(19.9.11) 


where  is  the  factor  loading  matrix. 
From  Eqs.  (19.9*8)  and  (19.9*11) 


„R 


iR  -  b.i  b!i 


(19.9.12) 


or,  in  general. 


-  iR  -  b.i  b'.i 


From  Eqs. . (19.9*8)  and  (19*9»13) 


i+la  i+la/  =  i+lR 


From  Eqs.  (19.9*9)/  (19*9.10),  and.  (19. 9.1*0 


s+lb.i 


iRSP.i 

//11  18  sP.i 


(19.9.13) 


(19-9.1*0 


(19.9.15) 


From  Eqs.  (19-9.1),  (19-9.13),  and  (19.9.15)  we  can  solve  iteratively 
for  b  From  Eq.  (19.9*15)  we  see  that,  for  any  iteration  s,  flb  ^  is  a 

rank  reduction  solution  for  Eq.  (19.9*13)/  irrespective  of  how  well  the 
solution  has  stabilized. 

We  let 

y  s  b* .  b  ,  (19.9.18) 

si  s.is.i 
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Then  from  Eqs.  (19.9.1)  and  (19.9.16) 


Let 


S  .  =  bty  _  b  . 
s  .1  s  .1  s  .1  n 


sW.i  -  i5SP.i 


From  Eqs.  (19.9.IO),  (19.9.14),  and  (19.9.18) 


a. 

s  1 


•/FIT* 


From  Eqs.  (19.9.15),  (19.9*18),  and  (19.9*19) 


s+lb.i 


8®i 


(19.9.17) 


(19* 9.18) 


(19.9.19) 


(19.9.20) 


The  computational  equations  then  for  the  b  .  are  given  by  Eqs.  (19.9.16) 
through  (19.9.20),  and  the  successive  jR's  are  calculated  from  Eq.  (19.9.13). 
The  variance  reduction  in  the  jR  matrix  accounted  for  by  the  £  approximation 
to  ^b^  is  obviously  given  by  j  in  Eq.  (19.9.I6).. 

19*9.2  Simultaneous  Factor  Matrix  from  Correlation  Matrix 
From  Section  18.4,2  we  have,  as  the  s  approximation  to  the  simultaneous 
varimax  factor  matrix, 


8  =  b^  -  b 
s  s  s 


1  b'  b 
s  s 


a'  sp 
C'  C 

S  S 


s+1 


C  Q  A  O' 
s  s  s  s^ 


asH 


(19.9.21) 

(19.9.22) 

(19.9.23) 

(19.9.24) 

(19.9.25) 


3 
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Prom  Eqs.  (19.9.22)  and  (19.9.23) 

P  « •'  /  ■  SQ  / 

From  Eqs.  (19.9.22),  (19.9.24),  and  (19.9.25) 
s+lb  "  a  a'  SP  SQ' 

We  let 


a  a'  =  R 


From  Eqs.  (19.9.26)  and  (19.9.28) 

.«  /  ■  »*>'  »■/ 

From  Eqs.  (19.9.27)  and  (19.9. 


s+1 


i  =  S  M  A1  0/ 

SK  S  3  s 


(19.9.26) 


(19.9.27) 


•  (19.9.28) 


(19.9.29) 


(19.9.30) 


M  =  R  (3 
s  8 


S°  -  sp/  SM 


From  Eqs.  (19.9.29)  and  (19.9.32) 


,Q  A  Q'  =  G 
3  3s  8 


From  Eqs.  (19.9.30)  and  (19.9.31) 


(19.9.31) 

(19.9.32) 

(19.9.33) 


M  Q  A*1  a' 

a  o^1  a  a  “ 


s+1  S  '  3"  S~  s' 


(19.9.34) 


Then  Eqs.  (19.9.21)  and  (19.9.31)  through  (19.9*34)  constitute  the  computation¬ 
al  equations  for  calculating  the  s  approximation  to  the  b  matrix.  That  any 
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approximation  gb  of  width  m  is  a  solution  for 


R  =  R  -  b  b' 
m  s  s 


(19.9.35) 


such  that  the  rank  of  R  is  m  less  than  the  rank  of  R,  can  be  readily  shown 

m  -  — 

as  follows.  Dropping  the  prescripts,  we  have  from  Eqs.  (19.9.29)  and  (19.9*30), 


b  =  R  p  (p;  R  g)  2 


(19.9.36) 


Substituting  Eq,  (19.9*36)  in  Eq.  ( 19.9.35) 

mR  =  R  -  R  P  (P'  R  3)'1  S'  R  (19.9.37) 

which  is,  of  course,  the  rank  reduction  form. 

The  iterations  may  proceed  until  tr  G  converges  to  the  desired  degree 

s 

of  decimal  accuracy. 

19.9.3  Successive  Factor  Varimax  from  Data  Matrix 
Given  the  data  matrix  Z  such  that 

R  =  Z'  Z  (19.9.38) 


The  successive  factor  varimax  from  the  correlation  matrix,  according  to 
Section  19.3.2,  is  given  by  the  set  of  equations 


s7i 


b'  b  . 
s  ,i  s  .i 


e*.i 


b(5)  .  b  u 

s  ,i  s  ,i  n 


(19.9.39) 

(19.9.40) 


sW.i  =  iR  s^.i 


(19.9.41) 
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and 


a.  =  fW.  FT 
si  y s  .1  sK.i 


W  • 

t  =  ®-ii 
s+l  .i  a 

s  i 


<4,R  =  .R  -  b  .  b; 

i+1  l  .i  .i 


From  Eqs,  (19.9.38),  (19.9.41),  (19.9.42)  and  (19.9.43) 

r  ./  .i 


s+lb.i 


,/sKi  Z'  2 


Let 


sU.i  -  ZsKi 


SY.l 


3U.i 


/  u'  u  . 

,/  s  .  is  .i 

From  Eqs.  (19.9.45),  (19.9.46),  and  (19.9*47) 


s+lb.i  "  Z'  sY.i 


Consider  now  the  residual  matrix 


!Z  -  ’’'.I 


From  Eqs.  (19.9.48)  and  (19.9.49) 


2Z  =  lZ  "  Y.l  Y'.l  1Z 


or 


(19.9.42) 

(19.9.43) 

(19.9.44) 

(19-9.45) 

(19.9.46) 

(19.9.47) 

(19.9.48) 

(19.9.49) 

(19.9.50) 


A 


2Z  =  (I  -  Ya  Y'a)  XZ 


(19.9.51) 
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t 


Then  in 

ffiZ  =  Z  -  Y  9  (19.9.59) 

the  rani:  of  Z  is  m  less  than  the  rank  of  Z.  Also,  it  should  now  he  obvious 

m  —  — 

that  Y  is  the  factor  score  matrix  corresponding  to  the  factor  loading  matrix 
b. 

19.9*4  Simultaneous  Varimax  from  the  Data  Matrix 
Consider  again  the  data  matrix  Z,  such  that 

R  =  Z'  Z  (19.9.60) 


The  simultaneous  varimax  matrix  from  the  correlation  matrix,  according  to 
Section  19.4,  is  given  by  the  set  of  equations 

D 


0  =  b 
e  s 


(3) 


b 


b'  b 
s  s 


(19.9.61) 


M  =  R  0 
s  s 


G  =  0'  M 

S  8  S 


(19.9.62) 

(19.9.63) 


s+l”  '  sMsG"  (19.9.6k) 

From  Eqs.  (19.9.60)  and  (19.9.62) 

M  =  Z*  Z  0  (19.9.65) 

S  5 

From  Eqs.  (19.9.60),  (19.9.62),  and  (19.9*65) 

G  =  0'  Z'  Z  0  (19.9.66) 

s  s  s 
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From  Eqs.  (19.9.62)  through  (19.9.66) 

s+1b  =  2'  Z  (;  Z'  Z  flP)“*  (19.9.67) 

Let 

Z  SP  =  su  (19.9.68) 

Let 

sU  =  sP  sA'  sQ'  (19.9.69) 

From  Eqs.  (19.9.66)  and  (19.9.68) 

SG  =  gU'  gU  (19.9.70) 

From  (19.9.60)  and  (19.9*70) 

8°  -  .«  / 

Let 

Y  =  U  G*^  (19.9.72) 

o  o  9 

From  Eqs.  (l9.9.?l)  and  (19.9.72) 

s*  •  s°  s«  Z1  s4'  (19-9-75) 

From  Egs.  (19.6. 67)  through  (19.9.73) 

-  z'  .1  09.9.1V 

We  can  then  solve  for  successive  approximations  to  b  and  Y  by  Eqts.  (19.9*61), 
(19.9.68),  (19.9.70),  (19.9.71),  and  (19.9.73). 


(19.9.75) 


l4l 


From  Eqs.  (19-9.69)  and  (19.9.73) 

Y  =  P  Q' 

s  s  s 

hence 

SY'  3*  =  I  (19.9.76) 

Let 

mz  =  Z  -  Y  b'  (19.9.77) 

From  Eqs.  (19. 9.74)  and  (19.9*77) 

mZ  =  Z-YY'Z  (19.9.78) 

or 

/  =  (I-YY')Z  (19.9.79) 

From  Eqs.  (19.9.60),  (19.9.74),  (19.9-76),  and  (19.9*79) 

Z'  Z  =  R  -  b  b'  (19.9.80) 

mm  s  s 

hence 

„»  =  R  .  sb  V  (19.9.&1) 


In  Section  19.9*2  we  proved  the  rank  of  is  m  less  than  the  rank  of  R. 

Hence  the  rank  of  ^Z  in  Eq.  (19.9.77)  is  m  less  than  the  rank  of  Z  for  any 
approximation  a. 

19.9.5  Successive  General  Varimax  Vectors 

The  direct  solutions  for  the  successive  factor  general  varimax  differ 


from  the  solutions  which  maximize  the  variance  of  squared  factor  loadings 


only  in  the  calculation  of  the  8  vectors. 

S  *1 


2  mg  -  1 


where 


(19.9.82) 


(19.9.83) 


and  m1  and  m g  are  both  integers  greater  than  0. 


,/ 

6  =  b(S&-l)  .  b(k-D  s  .i 

sp.i  s  .i  s  ,i  n 


(19.9.84) 


Fran  Eq.  (19.9.82) 


2  k  -  1 


2  (2  n»i  -  nig)  +  1 

2  ni2  -  1 


2  (m  -  m  )  +  1 
k  -  1  =  - ± - £ - 


2  mg  -  1 


(19.9.85) 


(19.9.86) 


If  we  wish  to  calculate  the  successive  general  varimax  vectors  directly 
from  the  correlation  matrix,  we  use  the  same  equations  as  in  Section  19.3.2, 
with  the  exception  of  the  6  .  vector  indicated  in  Eq.  (19.9.84). 

If  we  wish  to  calculate  the  successive  general  varimax  vectors  directly 
from  the  data  matrix,  we  use  the  same  equations  as  in  19.5*2,  with  the  excep¬ 
tion  of  the  8  vector  which  is  now  given  by  Eq.  (19.9.84). 

19.9.6  Simultaneous  General  Varimax  Factor  Matrix 

The  rationale  for  the  simultaneous  general  varimax  directly  from  the 


correlation  or  data  matrices  is  the  same  as  for  the  special  case  of  the 


squared  factor  loadings,  except  that  the  S  matrix  is  different  for  values 

5 

of  k  other  than  2.  The  general  expression  for  3  is  given  by 

s 


matrix. 


CHAPTER  20 

*  FACTOR  SCORE  MATRICES 

20 . 1  Introduc  ti on 

Over  the  past  ko  years  a  vast  amount  of  attention  has  been  given  to 
the  factor  analysis  of  correlation  matrices.  In  this  book  we  have  already 
devoted  a  large  number  of  chapters  to  various  methods  for  getting  factor 
loading  matrices  from  correlation  matrices.  We  saw  in  Chapter  k  how  we  may 
view  the  general  factor  analysis  problem  as  one  of  approximating  a  data 
matrix  by  another  of  lower  rank.  We  saw  that  the  problem  viewed  in  this 
way  is  one  of  finding  two  basic  matrices  whose  major  product  is  in  some 
sense  a  satisfactory  approximation  to  the  data  matrix.  The  factors  of 
this  major  product  have  a  common  order  much  less  than  the  smaller  order  of 
s  the  data  matrix,  and  therefore  the  rank  of  the  product  is  equal  to  the 

common  order  of  its  factors. 

Again  in  Chapter  4  we  saw  how  we  may  regard  the  postfactor  of  this 
product  as  the  transpose  of  the  factor  loading  matrix.  We  also  saw  how 
we  may  regard  the  prefactor  as  the  factor  score  matrix.  Therefore,  the 
factor  score  matrix  postmultiplied  by  the  transpose  of  the  factor  loading 
matrix  yields  the  lower  rank  approximation  to  the  data  matrix.  In  this 
formulation  of  the  problem,  the  communality  problem  does  not  appear.  There 
seems  to  be  no  clear  justification  for  considering  the  communality  concept 
when  we  view  factor  analysis,  not  as  a  method  of  factoring  the  correlation 
or  covariance  matrix,  but  rather  as  one  of  factoring  the  data  matrix. 
Guttman  (1955)  has  discussed  an  interesting  exception,  which  is,  however, 
beyond  the  scope  of  this  book. 

t  In  any  case,  with  all  of  the  attention  given  to  the  solutions  for 


factor  loading  matrices,  very  little  has  been  devoted  to  the  derivation  of 
factor  score  matrices.  This  is  especially  curious  inasmuch  as  the  scienti¬ 
fic,  logical,  and  philosophical  status  of  the  factor  score  matrix  would  ap¬ 
pear  to  be  at  least  as'  respectable  as  that  of  the  factor  leading  matrix. 

Some  would  argue,  of  course,  that  the  factor  loading  matrix  is  of  more  funda 
mental  importance  because  it  enables  us  to  identify  or  define  the  fundament¬ 
al  variables  of  a  particular  discipline.  Certainly  there  is  something  to 
be  said  for  this  point  of  view,  if  one  regards  the  major  objective  of  the 
simple  structure  or  transformation  technique  as  one  of  finding  factor  load¬ 
ings  which  are  relatively  invariant  from  one  sample  of  entities  and  attri¬ 
butes  to  another. 

From  a  philosophical  point  of  view,  these  considerations  may  justify 
greater  interest  in  the  factor  loading  matrix.  However,  from  a  purely  form¬ 
al  point  of  view,  considering  only  the  mathematics  involved,  there  is  no 
reason  to  be  more  interested  in  the  factor  loading  matrix  than  in  the  factor 
score  matrix.  Considering  the  model  in  which  the  major  product  of  these 
two  matrices  approximates  the  data  matrix,  there  is  actually  nothing  in  the 
mathematics  of  the  model  which  would  in  some  sense  give  higher  status  to 
the  postfactor  than  to  the  prefactor. 

This  statement  is  even  more  cogent  if  we  recall  Chapters  13  and  15 
dealing  with  linear  transformations  involving  both  scale  and  origin,  which 
may  be  applied  to  both  the  right  and  the  left  hand  sides  of  the  data  matrix. 
We  have  seen  that  these  operations  can  materially  affect  the  results  of  a 
factor  analysis,  and  that  the  traditional  practice  of  metricizing  the  data 
matrix  by  attributes  rather  than  by  entities,  is  more  or  less  arbitrary.  In 
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any  case,  the  problem  of  finding  the  left  hand  factor  of  the  matrix  product 
which  purports  to  approximate  the  data  matrix  appears  to  merit  considerably 
more  attention  than  it  has  received  in  the  past. 

But  even  the  attention  which  the  problem  has  received  seems  to  have 
caused  as  much  confusion  as  clarification  for  the  central  issues  involved. 
These  are  actually  rather  simple,  if  one  does  not  become  unduly  involved 
with  the  red  herrings  of  the  communal! ty  and  specificity  problems.  One 
need  only  examine  the  bewildering,  even  if  at  times  ingenious,  traditional 
discussions  of  the  factor  measurement  problem  to  realize  that  they  have 
often  strayed  far  from  the  solid  ground  of  the  data  matrix. 

Some  investigators  have  argued  that  factor  scores  give  no  more  infor¬ 
mation  than  do  the  measures  from  which  they  are  derived,  and  that  there¬ 
fore,  at  best  these  scores  are  of  more  theoretical  than  practical  interest. 
Unfortunately,  these  investigators  have  asked  the  wrong  question.  Instead 
of  asking  whether  the  factor  scores  give  more  information  than  the  original 
data  scores,  they  should  have  asked  whether  the  original  data  scores  give 
more  information  than  the  factor  scores.  If  one  asks  this  latter  question, 
he  may  conclude  that  the  data  matrix  may  yield  not  only  relevant  or  system¬ 
atic  information,  but  also  random  or  unreliable  information. 

One  may  then  regard  the  lower  rank  data  matrix  approximation  model  as 
a  procedure  for  eliminating  from  the  data  matrix  random  or  irrelevant  vari¬ 
ance.  Horst  (19^1)  has  utilized  the  factor  score  matrix  to  reduce  the  ef¬ 
fect  of  this  variance.  The  method  has  never  received  wide  attention.  Leiman 
(1951)  has  applied  this  conception  of  factor  analysis  objectives  to  experi¬ 
mental  data.  He  has  found  that,  by  the  use  of  lower  rank  approximation 
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matrices  to  data  matrices,  one  may  obtain  multiple  regression  parameters 
which  hold  up  better  on  cross-validation  than  when  the  data  matrices  are 
employed  directly  in  the  traditional  methods.  A  much  more  extensive  applica¬ 
tion  of  factor  score  matrices  and  the  lower  rank  approximation  model  has 
been  made  by  Burket  (1964).  In  his  work  it  is  clear  that  for  prediction 
purposes  the  lower  rank  approximation  procedures  have  a  definite  advantage 
over  the  conventional  multiple  regression  procedures.  This  application  of 
the  factor  score  concept  will  be  considered  in  more  detail  in  Chapter  23. 

Here  we  will  present  various  types  of  methods  for  calculating  factor  score 
matrices. 

20.2  Kinds  of  Factor  Score  Solutions 

We  may  classify  the  various  kinds  of  solutions  for  factor  score  matrices 
to  correspond  with  the  procedures  for  getting  factor  loading  matrices  which 
we  have  discussed  in  the  previous  chapters.  The  solutions  will  be  presented 
under  the  headings  of  the  centroid  factor  score  matrix,  the  multiple  group 
factor  score  matrix,  the  principal  axis  factor  score  matrix,  the  least  square 
factor  score  matrix,  and  the  image  analysis  factor  score  matrix. 

20.2.1  The  Centroid  Factor  Score  Matrix.  The  calculation  of  a  centroid 
factor  score  matrix  directly  from  the  data  matrix  has  already  been  explicit¬ 
ly  considered  in  Chapter  12.  In  this  process,  we  arrive  successively  at 
factor  loading  vectors  and  factor  score  vectors.  However,  in  this  chapter 
we  shall  present  the  calculation  of  a  factor  score  matrix  based  on  a  pre¬ 
vious  calculation  of  the  centroid  factor  loading  matrix.  This  factor  load¬ 
ing  matrix,  together  with  a  matrix  of  sign  vectors,  yields  a  transformation 
matrix  which,  when  applied  to  the  data  matrix,  gives  the  centroid  factor 
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score  matrix. 

20.2.2  The  Multiple  Group  Factor  Score  Matrix.  We  shall  see  that 
by  the  use  of  a  binary  grouping  matrix  we  can  also  calculate  a  multiple 
group  factor  score  matrix  directly  from  the  data  matrix.  As  a  matter  of 
fact,  it  is  easier  to  use  the  multiple  group  method  directly  on  the  data 
matrix  than  it  is  to  use  the  centroid  method,  because  in  the  centroid 
method  we  must  have  a  matrix  of  sign  vectors,  and  this  ordinarily  becomes 
available  only  with  a  successive  factor  solution.  In  the  centroid  method, 
it  will  be  recalled  that  residual  matrices  are  calculated,  and  with  each 
residual  matrix  one  iterates  to  the  optimal  sign  vector. 

On  the  other  hand,  in  the  case  of  the  multiple  group  method,  one  pre¬ 
sumably  has  an  a  priori  binary  grouping  matrix.  For  this  reason  one  need 
not  go  through  the  actual  calculation  of  the  multiple  group  factor  loading 
matrix  before  calculating  the  multiple  group  factor  score  matrix. 

20.2.3  The  Principal  Axis  Factor  Score  Matrix.  Perhaps  the  simplest 
and  most  elegant  of  all  of  the  methods  for  getting  factor  score  matrices  is 
the  basic  structure  method.  As  we  have  seen  in  Chapter  4,  the  principal 
axis  or  basic  structure  type  solution  yields  the  least  square  approximation 
to  the  data  matrix  for  any  specified  rank  of  the  approximation  matrix.  We 
have  seen  that  the  principal  axis  or  basic  structure  factor  loading  matrix 
is  both  a  rank  reduction  solution  and  a  least  square  approximation  to  the 
correlation  or  covariance  matrix.  It  is  easy  to  show  that  the  first  m 
vectors  of  the  left  basic  orthonormal  matrix  of  the  data  matrix  yield  pre¬ 
cisely  the  principal  axis  factor  score  matrix,  and  that  this  is  an  ortho¬ 
gonal  matrix. 


149 


20.2.4  The  Least  Square  Factor  Score  Matrix.  Having  given  some  arbit¬ 
rary  factor  loading  matrix,  whether  centroid,  multiple  group,  or  principal 
axis,  we  may  wish  to  determine  that  factor  score  matrix  which,  when  post- 
multiplied  by  the  transpose  of  the  factor  loading  matrix,  yields  a  product 
which  is  the  least  square  approximation  to  the  data  matrix.  This  means  that 
the  sums  of  squares  of  elements  of  the  residual  matrix  will  be  a  minimum. 

This  will  in  general  be  true,  as  we  have  seen,  for  the  principal  axis  method. 
We  can  also  find  factor  score  matrices  for  the  centroid  and  the  multiple 
group  methods  which  are  least  square  solutions  to  the  data  matrix.  As  a 
matter  of  fact,  for  any  arbitrary  basic  matrix  of  width  equal  to  the  number 
of  attributes  and  of  height  equal  to  the  number  of  factors,  we  can  find  what 
particular  vertical  matrix,  postmultiplied  by  the  transpose  of  the  arbitrary 
factor  matrix,  yields  a  product  which  is  the  best  approximation  to  the  data 
matrix  in  the  least  square  sense. 

20.2.5  The  Image  Analysis  Factor  Score  Matrix.  To  our  knowledge,  no 
detailed  analysis  for  the  calculation  of  the  factor  score  matrix  from  the 
image  type  factor  loading  matrix  has  been  previously  presented.  Harris  (1962) 
has  given  an  interesting  theoretical  analysis  of  this  problem  in  a  recent 
paper.  The  image  analysis  approach  implies  a  transformation  of  the  data 
matrix.  It  is  therefore  of  interest  to  see  what  procedures  are  appropriate 
in  the  calculation  of  factor  score  matrices  based  on  these  image  factor  load¬ 
ing  matrices  and  the  transformed  data  matrix. 

20.3  The  Centroid  Factor  Score  Matrix 

20.3.1  Characteristics  of  the  Method.  We  have  indicated  in  the  pre¬ 
vious  section  that  the  centroid  factor  loading  matrix  may  be  used  in  the 
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solution  of  a  factor  score  matrix,  such  that  the  major  product  of  the  two 
will  give  the  best  least  square  approximation  to  the  data  matrix.  We  shall, 
however,  consider  here  only  a  particular  type  of  centroid  factor  score  solu¬ 
tion.  In  this  solution  the  factor  score  matrix  is  an  orthogonal  matrix. 

It  has  been  repeatedly  said  or  implied  that  the  centroid  solution 
yields  orthogonal  factors.  But  like  much  of  the  discussion  about  ortho¬ 
gonal  factors,  the  definition  of  orthogonal  factors  has  been  vague.  We 
shall  insist  on  using  the  term  orthogonal  only  with  respect  to  vectors.  We 
shall  insist  that  the  term  orthogonality  is  not  useful  unless  considered 
in  this  connection.  By  saying  that  two  vectors  are  orthogonal,  we  simply 
mean  that  their  minor  product  is  0. 

The  solution  for  the  centroid  factor  score  matrix  which  we  shall  con¬ 
sider  does  yield  factor  scores  such  that  the  minor  product  of  any  pair  of 
factor  score  vectors  taken  from  the  factor  score  matrix  will  be  0. 

A  further  characteristic  of  this  method  is  that  the  solution  is  a  rank 
reduction  solution.  That  is,  the  solution  is  such  that,  when  the  major 
product  of  the  factor  score  matrix  by  the  factor  loading  matrix  is  sub¬ 
tracted  from  the  data  matrix,  the  residual  matrix  is  of  rank  equal  to  the 
rank  of  the  data  matrix  less  the  number  of  factors. 

This  solution,  like  all  of  the  solutions  for  factor  score  matrices 
which  we  shall  consider,  involves  no  iterative  procedures.  In  this  respect 
it  is  relatively  simple  and  straightforward  computationally. 

The  methods  outlined  in  all  of  the  procedures  in  this  chapter  are  con¬ 
cerned  particularly  with  the  calculation  ef  transformation  matrices  which  may 
be  applied  to  data  matrices  to  convert  them  to  factor  score  matrices. 
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Therefore,  in  actual  practice  it  may  he  desirable  to  use  these  transforma¬ 
tion  matrices  on  data  matrices  other  than  those  from  which  the  factor  load¬ 
ing  matrices  were  calculated.  When  this  is  the  case,  the  resulting  factor 
score  matrices  cannot  he  expected  to  exhibit  precisely  the  same  character¬ 
istics  as  when  these  transformation  matrices  are  applied  to  the  original 
data  matrix. 

20.3.2  Computational  Equations 

20.3.2a  Definition  of  Notation 

a  is  the  centroid  factor  loading  matrix. 

L  is  the  matrix  of  sign  vectors. 

Y  is  the  centroid  factor  score  matrix. 

Z  is  the  normalized  data  matrix. 

20.3.2b  The  Equations 

t'  =  a'  L  (20.3.1) 

B  =  L  t'"1  (20.3.2) 

Y  =  Z  B  (20.3.3) 

20.3.3  Computational  Instructions.  We  begin  with  the  centroid  factor 
loading  matrix  a.  We  also  have  given  the  matrix  of  sign  vectors  used  in  the 
calculation  of  the  centroid  factor  loading  matrix,  which  we  designate  as  L. 

We  calculate  the  upper  triangular  matrix  as  indicated  in  Eq.  (20.3*1). 
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This  is  the  transpose  of  the  factor  loading  matrix  postmultiplied  by  the 
sign  matrix. 

Next  we  calculate  Eq.  (20.5.2),  which  is  the  matrix  of  sign  vectors 
postmultiplied  by  the  inverse  of  the  triangular  matrix  in  Eq.  (20.5.1). 

We  now  calculate  the  centroid  factor  score  matrix,  as  indicated  in  Eq. 
(20.5.5).  This  is  the  data  matrix  postmultiplied  by  the  transformation 
matrix  of  Eq.  (20.5.2). 

20.5.4  Numerical  Example.  We  begin  with  the  data  matrix  used  in 
Chapter  1$),  whose  corresponding  correlation  matrix  is  the  same  used  in  pre¬ 
vious  chapters.  The  data  matrix  is  repeated  for  convenient  reference  in 
Table  20.5.1. 

Table  20.5.2  gives  the  centroid  factor  loading  matrix  for  three  factors, 
calculated  from  the  correlation  matrix. 

Table  20.5.5  gives  the  matrix  of  sign  vectors  by  rows,  used  in  calcu¬ 
lating  the  centroid  matrix,  and  subsequently  in  the  calculation  of  the  factor 
score  matrix. 

Table  20.5.4  gives  the  centroid  factor  score  matrix  for  three  factors, 
calculated  by  means  of  Eqs.  (20.5*1)  through  (20.5.4). 

Table  20.5*5  gives  the  minor  product  moment  of  the  factor  score  matrix. 
As  proved  in  Section  20.8.1,  this  should  be  an  identity  matrix,  which  it  is 
within  limits  of  rounding  error. 
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Table  20.3*1  -  Normalized  Deviation  Data  Matrix 


0.128 

0.181 

0.121 

0.506 

0.857 

0.716 

0.280 

0.178 

0.216 

0.761 

0.710 

0.563 

-0.387 

-0.293 

-0.202 

0.261 

0.281 

0.013 

-0.030 

-0.016 

O.Oll 

0.H7 

-0.109 

-0.135 

0.610 

0.682 

0.661 

-0.280 

-0.351 

-0.326 

.  -0.023 

-0.109 

-0.186 

O.085 

0.091 

-O.65I 

-0.336 

-0.306 

-0.129 

-0.512 

-0.006 

-0.153 

-0.128 

0.056 

-0.121 

-0.276 

-0.321 

-0.271 

-0.370 

-0.225 

0.035 

0.129 

-0.116 

-0.121 

0.057 

-0.070 

-0.016 

0.006 

0.152 

-0.111 

-0.166 

-0.351 

-0.033 

-0.010 

-O.llO 

0.326 

-o.ool 

-0.258 

-0.091 

-0.110 

-0.200 

-0.101 

-0.303 

0.227 

-0.011 

0.029 

-0.102 

-0.125 

-0.086 

-0.106 

-0.071 

0.086 

0.057 

-0.003 

0.161 

0.073 

0.13I 

-0.082 

-0.067 

-0.009 

0.161 

0.106 

-0.121 

0.251 

-0.002 

0.227 

-0.072 

0.025 

0.156 

0.036 

-0.071 

-0.112 

0.212 

0.023 

0.192 

-0.118 

-0.119 

0.011 

Table  20.3.2 

-  Centroid  Factor  Loading  Matrix  by  Rows  for  Three  Factors 

0.659 

0.681 

0.730 

0.617 

0.512 

0.588 

0.637 

0.606 

0.708 

0.157 

0.152 

0.283 

-0.601 

-0.709 

-0.663 

0.271 

0.316 

0.161 

O.I56 

0.139 

0.151 

-0.209 

0.091 

0.116 

-0.179 

-0.188 

-0.378 

Table  20.3.3 

-  Sign  Matrix  by  Rows  for  Three  Factors 

1.  1.  - 

1.  1. 

1.  1. 

1.  1. 

1.  • 

1.  1. 

1.  -1. 

-1.  -1. 

1.  1. 

1. 

1.  1. 

1.  -1. 

1.  1. 

-1.  -1. 

-1. 

Table  20.3*1  -  Normalized  Centroid  Factor  Score  Matrix  by  Columns 


0.611 

-0.525 

0.175 

0.307 

0.719 

0.162 

0.316 

0.329 

-0.810 

-0.301 

-0.108 

-0.191 

-0.393 

0.007 

0.019 

-0.321 

-0.003 

-0.015 

-0.150 

0.011 

0.105 

-0.151 

0.013 

0.211 

-0.096 

0.015 

-o.ool 

0.061 

-0.133 

0.083 

0.121 

-0.123 

-0.031 

o.ool 

-0.228 

-0.011 

Table  20.3.5  -  Minor  Product  Moment  of  Centroid  Factor  Score  Matrix 

0.997  -0.001  0.002 

-0.001  0.999  0.000 

0.002  0.000  1.001 
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20.4  The  Multiple  Group  Factor  Score  Matrix 

20.4.1  Characteristics  of  the  Solution.  The  solution  for  the  multiple 
group  factor  scores,  as  already  indicated,  does  not  first  require  the  calcu¬ 
lation  of  the  multiple  group  factor  loading  matrix.  If  we  have  the  binary 
grouping  matrix,  it  can  be  applied  directly  to  the  correlation  matrix  to 
yield  a  transformation  matrix  which,  when  applied  to  the  data  matrix,  gives 
the  factor  score  matrix.  As  in  the  case  of  the  centroid  method,  this  solu¬ 
tion  is  not  a  least  square  solution  in  the  sense  that  the  major  product  of 
the  factor  score  and  factor  loading  matrices  gives  the  best  least  square 
fit  to  the  data  matrix.  However,  it  does  yield  a  factor  score  matrix  which 
is  orthonormal  and  of  rank  reduction  form. 

20.4.2  Computational  Equations 

20.4.2a  Definition  of  Notation 

R  is  the  correlation  matrix, 
f  is  a  binary  grouping  matrix, 
t  is  a  lower  triangular  matrix. 

Y  is  the  multiple  group  factor  score  matrix. 

Z  is  the  normalized  data  matrix. 

20.4.2b  The  Equations 
G  =  f'  Ef 
t  t'  =  G 


(20.4.1) 

(20.4.2) 
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B  =  f  t'*1  (20.4.3) 

Y  .’ZB  (20.4.4) 

20.4.3  Computational  Instructions.  We  assume  that  the  correlation 
matrix  R  and  a  binary  grouping  matrix  f  are  given.  We  then  calculate  the 
matrix  in  Eq.  (20.4.1).  This  is  the  correlation  matrix  postmultiplied  by 
the  binary  grouping  matrix  and  premultiplied  by  its  transpose. 

Next  we  calculate  the  triangular  factors  of  the  matrix  of  Eq.  (20.4.1), 
as  indicated  in  Eq.  (20.4.2). 

In  Eq.  (20.4.3)  we  postmultiply  the  binary  grouping  matrix  by  the  in¬ 
verse  of  the  upper  triangular  factor  in  Eq.  (20.4.2). 

The  multiple  group  factor  score  matrix  is  indicated  in  Eq.  (20.4.4). 

This  is  the  data  matrix  postmultiplied  by  the  transformation  matrix  of  Eq. 
(20.4.3).  The  minor  product  moment  of  this  matrix  is  shown  in  Section  20.3.2 
to  be  the  identity  matrix. 

20.4.4  Numerical  Example.  We  use  the  same  data  matrix  and  correlation 
matrix  as  in  the  previous  section. 

Table  20.4.1  gives  the  binary  grouping  matrix  by  rows  for  three  factors. 

Table  20.4.2  gives  the  normalized  multiple  group  factor  score  matrix  by 
rows  for  three  factors.  The  matrix  was  calculated  by  means  of  Eqs.  (20.4.1) 
through  (20.4.4). 

Table  20.4.3  gives  the  minor  product  moment  of  the  multiple  group  factor 
score  matrix.  Within  rounding  error  this  is  an  identity  matrix,  as  it  should 
be  according  to  Section  20.8.2.  In  this  sense  the  multiple  group  factors  may 
be  said  to  be  orthogonal,  but  only  if  the  factor  scores  are  calculated  in 
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this  manner. 

The  same  may  also  be  said  for  the  centroid  factors  when  the  factor 
score  matrix  is  calculated  according  to  Section  20.5* 
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Table  20.4.1  -  Binary  Grouping  Matrix  by  Rows  for  Three  Factors 

1.  1.  1.  0.  0.  0.  0.  0.  0. 

0.  0.  0.  1.  X.  X.  0.  0.  0. 

0.  0.  0.  0.  0.  0.  1.  X.  X. 


Table  20.4.2  -  Normalized  Multiple  Group  Factor  Score  Matrix  for  Three 
Factors 


0.262 

0.767 

0.007 

0.743 

-0.470 

0.022 

-0.022 

-0.033 

0.902 

-0.544 

-0.062 

-0.037 

-0.385 

-0.203 

0.009 

-0.315 

-0.161 

-0.010 

-0.010 

-0.107 

-0.216 

0.063 

-0.147 

-0.312 

-0.032 

-0.071 

-0.086 

0.050 

0.133 

-0.126 

0.052 

0.168 

-0.016 

-0.065 

0.187 

-0.128 

Table  20.4.3  -  Minor  Product  Moment  of  Multiple  Group  Factor  Score  Matrix 

1.000  0.000  -0.003 
0.000  1.000  -0.001 
-0.003  -0.001  1.000 


20.5  The  Principal  Axis  Factor  Score  Matrix 


20.5.1  Characteristics  of  the  Method.  As  indicated  in  the  previous 
discussion  on  kinds  of  methods,  the  principal  axis  solution  is  the  simplest 
of  the  methods,  if  the  basic  structure  or  principal  axis  factor  loading 
matrix  is  already  available.  One  of  the  most  important  advantages  of  the 
principal  axis  method  is  that  it  gives,  at  the  same  time,  a  least  square, 

a  rank  reduction,  and  an  orthogonal  solution  for  the  factor  score  matrix. 

20.5.2  Computational  Equations 
20.5.2a  Definition  of  Notation 

a  is  the  principal  axis  factor  loading  matrix. 

B  is  the  basic  diagonal  of  the  correlation  matrix. 

Y  is  the  principal  axis  factor  score  matrix. 

Z  is  the  normalized  data  matrix. 

20.5.2b  The  Equations 


a  B'1 

(20.5.1) 

Z  B 

(20.5.2) 

20.5*3  Computational  Instructions.  The  computational  instructions 
for  the  principal  axis  factor  score  matrix  are  very  simple.  We  begin  with 
the  factor  loading  matrix  a  and  the  basic  diagonal  B.  Eq.  (20.5.1),  then, 
directly  gives  the  transformation  matrix.  This  is  simply  the  factor  loading 
matrix  postmultiplied  by  the  inverse  of  the  B  diagonal. 


The  factor  score  matrix  is  then  calculated  in  the  usual  manner,  as  in¬ 
dicated  in  Eq.  (20.5.2). 

Hotelling  (1953)  published  this  solution  for  the  factor  score  matrix. 
However,  it  does  not  seem  to  be  well  known  and  has  not  been  used  extensively. 

21.5.4  Numerical  Example.  In  this  example  we  use  the  same  data  and 
correlation  matrices  as  in  the  previous  sections. 

Table  20.5.1  gives  the  first  three  basic  diagonals  of  the  correlation 
matrix,  as  found  in  early  chapters  giving  basic  structure  solutions. 

Table  20.5.2  gives  the  first  three  principle  axis  factor  vectors  by 
rows,  as  found  in  previous  chapters,  for  example.  Chapters  7  and  8. 

Table  20.5.3  gives  the  normalized  principal  axis  factor  score  matrix 
for  three  factors,  as  calculated  from  Eqs.  (20. 5.1)  and  (20.5.2). 

Table  20.5.4  gives  the  minor  product  moment  of  the  principal  axis 
factor  score  matrix.  This  is  the  identity  matrix  to  within  rounding  error. 
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Table  20.5.1  -  First  Three  Basic  Diagonals  of  Correlation  Matrix 
5.7^9  2.050  1.331 


Table. 20. 5. 2 


0.717 

0.493 

0.350 


-  Principal  Axis  Factor  Loading  Matrix 
Factors 

by  Rows 

for  Three 

0.740  0.773 

0.556  0.463 

0.518 

0.640 

0.615 

0.715 

0.478  0.296 

-0.649  -0.744 

-0.694 

0.080 

0.166 

-0.034 

0.322  0.406 

0.068  0.181 

0.188 

-0.588 

-0.621 

-0.369 

Table  20.5.3  -  Normalized  Principal  Axis  Factor  Score  Matrix  for  Three 
Factors 


0.555 

-0.569 

0.179 

0.386 

0.767 

0.205 

0.325 

0.092 

-0.825 

-0.328 

-0.065 

-0.198 

-0.403 

0.001 

-0.146 

-0.325 

-0.023 

-0.093 

-0.140 

0.053 

0.200 

-0.129 

0.107 

0.317 

-O.O88 

0.037 

0.049 

0.052 

-0.098 

0.143 

0.114 

-0.107 

0,052 

-0.014 

-0.195 

0.109 

Table  20.5.4  -  Minor  Product  Moment  of  Principal  Axis  Factor  Score  Matrix 


O.998 

-0.001 

0.002 


-0.001 

1.000 

0.000 


0.002 

0.000 

1.002 
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20.6  The  Least  Square  Factor  Score  Matrix 

20.6.1  Characteristics  of  the  Method.  This  method  is  not  mutually 
exclusive  of  those  previously  considered.  It  may  he  applied  to  any  factor 
loading  matrix  such  as  the  centroid,  the  multiple  group,  or  the  principal 
axis.  When  applied  to  the  principal  axis  factor  matrix  it  yields  precisely 
the  solution  given  in  the  preceding  section.  The  least  square  solution 
yields  a  factor  score  matrix  such  that,  when  the  major  product  of  this 
matrix  and  the  factor  loading  matrix  is  subtracted  from  the  data  matrix, 
the  sum  of  squares  of  elements  in  the  residual  matrix  is  a  minimum.  This 
solution,  as  all  least  square  solutions  in  general,  can  be  shown  to  be  a 
rank  reduction  solution.  The  left  arbitrary  multiplier,  however,  is  some- 

*  what  more  involved  than  in  other  methods,  as  can  be  seen  from  Section  20.9.4. 

In  general,  also,  the  computations  are  somewhat  more  involved  than  they  are 
for  the  solutions  we  have  already  discussed. 

20.6.2  Computational  Equations 

20.6.2a  Definition  of  Notation 

a  is  an  arbitrary  factor  loading  matrix. 

Z  is  the  normalized  data  matrix. 

Y  is  the  arbitrary  factor  score  matrix. 

20.6.2b  The  Equations 

G  =  a'  a 

B  =  a  G'1 
=  Z  B 


Y 


(20.6.1) 

(20.6.2) 

(20.6.5) 
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20.6.3  Computational  Instructions.  We  begin  with  any  arbitrary  factor 
loading  matrix. 

Eg,.  (20.6.1)  gives  the  minor  product  moment  of  the  arbitrary  factor 
loading  matrix. 

The  next  step  is  indicated  by  Eg.  (20.6.2),  which  is  the  factor  load¬ 
ing  matrix  postmultiplied  by  the  inverse  of  the  matrix  in  Eq.  (20.6.1).  This 
is  the  matrix  which  transforms  the  data  matrix  to  the  factor  score  matrix. 

Eq.  (20.6.3)  shows  the  least  square  factor  score  matrix  as  the  product 
of  the  data  matrix  postmultiplied  by  the  transformation  matrix  of  Eq. 
(20.6.2). 

20.6.4  numerical  Example.  We  use  the  same  data  and  correlation  matri¬ 
ces  as  in  the  previous  sections.  We  also  use  the  centroid  factor  matrix  of 
Section  20.3.4. 

Table  20.6.1  shows  the  minor  product  moment  matrix  of  the  centroid 
factor  matrix  for  three  factors. 

Table  20.6.2  gives  the  inverse  of  the  matrix  in  Table  20.6.1. 

Table  20.6.3  gives  the  product  of  the  natural  order  of  the  matrix  in 

Table  20.6.1  postmultiplied  by  the  matrix  of  Table  20.6.2.  This  gives  the 
matrix  for  transforming  the  data  matrix  to  the  least  square  factor  score< 
matrix. 

Table  20.6.4  gives  the  least  square  factor  score  matrix. 

Table  20.6.5  gives  the  minor  product  moment  of  the  least  square  factor 

score  matrix.  This  is  not  an  identity  matrix,  nor  should  it  be  so,  unless 
the  factor  loading  matrix  consists  of  basic  structure  factors. 
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Table  20.6.1  -  Minor  Product  Moment  of  Centroid  Factor  Loading  Matrix 
for  Third  Factors 


3.730 

0.170 

0.052 

0.170 

2.017 

O.I56 

0.052 

0.156 

1.280 

Table  20.6. 

,2  -  Inverse  of  Minor  Product  Moment  of  Factor  Loading  Matrix 

0.269 

-0.022 

-0.008 

-0.022 

0.502 

-0.060 

-0.008 

-0.060 

0.789 

Table  20.6. 

3  -  Matrix  for  Transforming  Data  Matrix  to  Factor  Score  Matrix 

0.164 

0.188 

0.327 

0.171 

0.185 

0.313 

0.187 

0.099 

0.333 

0.181 

-0.303 

-0.134 

0.161 

-0.374 

0.112 

0.172 

-0.353 

0.127 

0.169 

0.151 

-0.399 

0.160 

0.190 

-0.411 

0.190 

0.090 

-0.314 

Table  20.6. 

4  -  Least  Square  Factor  Score  Matrix 

0.611 

-0.540 

0.100 

0.302 

0.731 

0.429 

0.319 

0.316 

-0.812 

-0.316 

-0.065 

-0.208 

-0.400 

-0.007 

-0.109 

-0.321 

-0.031 

-0.083 

-0.147 

-0.002 

0.174 

-0.143 

0.025 

.  0.299 

-0.092 

0.018 

0.037 

0.061 

-0.123 

0.112 

0.124 

-0.104 

0.012 

0.005 

-0.215 

0.040 

Table  20.6. 

,5  -  Minor  Product  Moment  of  Least  Square  Factor  Score  Matrix 

0.998 

-0.002 

0.004 

-0.002 

1.004 

0.004 

0.004 

0.004 

1.050 

v 
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20.7  The  Image  Analysis  Factor  Score  Matrix 

20.7.1  Characteristics  of  the  Method.  When  the  image  analysis  approach 
to  factor  analysis  is  used,  we  may  employ  any  of  the  factoring  methods  dis¬ 
cussed  so  far:  the  principal  axis,  the  group  centroid,  the  centroid,  the 
multiple  group,  or  other  methods.  Furthermore,  we  may  also  use  any  of  the 
metricizing  methods  of  Chapters  13  and  15 .  We  shall  in  our  description  of 
computational  procedures  indicate  a  scaling  diagonal.  In  particular,  this 
may  be  an  identity  matrix. 

20.7*2  Computational  Equations 
20.7.2a  Definition  of  Notation 

R  is  the  correlation  matrix. 

D  is  an  attribute  scaling  matrix. 

t  is  a  triangular  matrix. 

Y  is  the  image  factor  score  matrix. 

Z  is  the  normalized  data  matrix. 

20.7.2b  The  Equations 


M  = 

(I  -  R'1  D*-l)  D 

(20.7.1) 

0  = 

D  (R  -  2  D*-l  +  D*-l  R'1  DjJ-i)  D 

(20.7.2) 

C  = 

L'  G  L 

(20.7.5) 
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t  t' 

=  C 

(20.7.4) 

B  = 

M  (L  t'"1) 

(20.7.5) 

Y  = 

Z  B 

(20.7.6) 

20.7.3  Computational  Instructions.  We  shall  assume  the  correlation 
matrix  given.  We  then  calculate  a  matrix  M  as  indicated  in  Eq.  (20.7.1). 

This  will  be  recognized  as  the  matrix  which  transforms  the  data  matrix  to 
the  image  of  the  data  matrix.  The  matrix  on  the  extreme  right  of  the  right 
hand  side  of  the  equation  is  a  scaling  diagonal.  It  may  be  chosen  accord¬ 
ing  to  one  of  the  methods  suggested  in  Chapter  16,  or  it  may  be  determined 
according  to  the  self -scaling  procedures  of  Chapter  15 . 

We  then  calculate  the  image  covariance  scaled  matrix,  as  in  Eq.  (20.7.2). 
The  part  in  parentheses  on  the  right  hand  side  of  this  equation  will  be  recog¬ 
nized  as  the  standard  covariance  image  matrix  of  Guttman  (1953).  It  is  pre- 
and  postmultiplied  by  the  diagonal  scaling  matrix  of  Eq.  (20.7.1). 

Next  we  calculate  the  C  matrix  in  Eq.  (20.7*3).  This  is  the  matrix  of 
Eq.  (20.7.2)  premultiplied  by  the  transpose  of  an  L  matrix  and  postmultiplied 
by  the  natural  order  of  this  matrix.  This  L  matrix  is  of  the  same  orde.y.  .as  the 
factor  loading  matrix,  which  presumably  has  been  obtained  from  the  G  matrix. 

In  particular,  it  may  be  a  binary  grouping  matrix,  a  matrix  of  sign  vectors 
for  the  centroid  method,  or  a  principal  axis  factor  loading  matrix  calculated 
from  the  G  matrix  of  Eq.  (20.7.2).  This  depends  on  which  particular  type  of 
factor  loading  matrix  one  has  calculated. 

Eq.  (20.7*4)  shows  a  triangular  factoring  of  the  matrix  in  Eq.  (20.7.3). 
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Next  the  B  matrix  is  postmultiplied  by  the  inverse  of  the  upper  tri¬ 
angular  factor  of  Eq.  (2Q.7.k),  and  then  the  matrix  of  Eq.  (20.7.1)  is  post- 
multiplied  by  this  product,  to  give  the  matrix  B  of  Eq.  (20.7.5). 

The  matrix  of  Eq.  (20.7*5)  is  the  transformation  matrix  which,  when  ap¬ 
plied  to  the  data  matrix  as  in  Eq.  (20.7*6),  yields  the  factor  score  matrix. 
This  factor  score  matrix  is  orthonormal  and  of  rank  reduction  form.  It  is 
not  a  least  square  solution,  unless  L  in  Eq.  (20.7.3)  happens  to  be  the  basic 
structure  or  principal  axis  factor  loading  matrix  for  the  covariance  matrix 
G  in  Eq.  (20.7.2). 

20,7.4  Numerical  Example.  We  use  the  same  data  and  correlation  matri¬ 
ces  as  in  the  previous  sections,  and  the  grouping  matrix  of  Table  20.4.1. 

*  The  identity  matrix  is  taken  as  the  scaling  diagonal. 

Table  20.7.1  gives  the  image  covariance  matrix. 

* 

Table  20.7.2  gives  the  image  factor  score  matrix  for  three  factors, 
as  calculated  by  means  of  Eqs.  (20.7.1)  through  (20.7.6). 

Table  20.7.3  gives  the  minor  product  moment  of  the  image  factor  score 
matrix.  This  is  an  identity  matrix  to  within  rounding  error,  as  it  should 
be. 
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Table  20.7.1  -  Image  Covariance  Matrix 


0.733 

0.688 

0.662 

0.120 

0.065 

0.101 

0.294 

0.302 

0.336 

0.688 

0.744 

0.666 

0.147 

0.067 

0.117 

0.316 

0.307 

0.358 

0.662 

0.666 

0.691 

0.215 

0.169 

0.239 

0.288 

0.305 

0.355 

0.120 

0.147 

0.215 

0.511 

0.488 

0.494 

0.218 

0.178 

0.285 

0.065 

0.067 

0.169 

0.488 

0.572 

0.486 

0.161 

0.081 

0.246 

0.101 

0.117 

0.239 

0.494 

0.486 

0.562 

0.160 

0.142 

0.262 

0.294 

O.316 

0.288 

0.218 

0.161 

0.160 

0.481 

0.396 

0.418 

0.302 

0.307 

0.305 

0.178 

0.081 

0.142 

0.396 

0.499 

0.388 

0.336 

0.358 

0.355 

0.285 

0.246 

0.262 

0.418 

0.388 

0.436 

Table  20.7*2  -  Image  Factor  Score  Matrix  for  Three  Factors 


0.265 

0.796 

-0.058 

0.734 

-0.490 

0.013 

0.018 

0.018 

0.881 

-0.359 

-0.112 

0.121 

*>0.383 

-0.170 

-0.019 

-0.316 

-0.106 

-O.O68 

-0.025 

-0.104 

-0.248 

0.019 

-0.113 

-0.331 

-0.034 

-0.078 

-0.051 

0.054 

0.102 

-0.115 

0.079 

0.111 

-0.003 

-0.053 

0.144 

-0.111 

Table  20.7.3  -  Minor  Product  Moment  of  Image  Factor  Score  Matrix 


1.000  0.000  -0.004 
0.000  1.000  -0.001 
•o.oo4  -o.ool  0.999 


% 
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20.8  Mathematical  Proofs 

20.8.1  The  Centroid  Factor  Score  Matrix 
Consider  any  basic  matrix  L  of  order  n  x  m. 


iR  L.i 

Aa R  L.i 


(20.8.1) 


where 


i+i*  •  iE  -  a.i  a'.i 


(20.8.2) 


a  ‘  (a.l  —  %> 


(20.8.3) 


We  can  prove,  by  the  methods  of  Chapter  5,  from  Eqs.  (20.8.1)  and  (20.8.2) 


1/  a  =  t 


(20.8.4) 


where  t  is  lower 
a 

Consider  now 


l'  r  l  =  ^  t* 


(20.8.5) 


b  =  ,R  L  t 


(20.8.6) 


From  Eqs.  (20.8.5)  and  (20.8.6) 
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t 


s 
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L'  b  =  (20.8.7) 

From  Eqs.  (20.8.2)  and  (20.8.3) 

m+1R  =  R  -  a  a'  (20.8.8) 

From  Eqs.  (20.8.1)  through  (20.8.4)  it  can  be  shown  that 

m+1RL  =  0  (20.8.9) 

From  Eqs.  (20.8.8)  and  (20.8.9) 

L'  RL  =  L'  a  a'  L  (20.8.10) 

From  Eqs.  (20.8.5),  (20.8.6),  and  (20.8.7) 

L'  RL  =  L'  b  b'  L  (20.8.11) 


From  Eqs.  (20.8.4),  (20.8.7),  (20.8.10),  and  (20.8.11) 

ta  =  (20.8.12) 

From  Eqs.  (20.8.4),  (20.8.7),  and  (20.8.12) 

L'  a  =  L'  b  (20.8.13) 

If  m  =  n  and  L  is  nonsingular,  we  have  from  Eq.  (20.8.13)  that  a  =  b.  But 

from  Eqs.  (20.8.1)  and  (20.8.2)  the  solution  for  any  a  ^  is  independent  of 

the  solution  for  any  a  Also,  from  Eq.  (20.8.6)  the  solution  for  any 

b  .  is  independent  of  the  solution  for  any  b  .  .  .  Hence,  in  general 
•1  • 


=  b 


a 


(20.8.14) 
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Consider  then  the  n  x  m  matrix  L  of  sign  vectors  for  the  centroid 


■factor  solution  such  that 


.  ,,R  =  .R  -  a  .  a' 

l+l  1  .i  .i 


(20.8.15) 


where 


iaL.i 

A'.i ®  L.i 


(20.8.16) 


Fr«n  Eqs.  (20.8.3)  and  (20.8.4)  we  can  express  the  centroid  factor  matrix 
as  a  function  directly  of  the  correlation  matrix  by 


a  =  R  L  tr 


(20.8.17) 


where 


tt'  =  L'  R  L 


(20.8.18) 


We  may  now  let  L  and  L1  Z1  be  the  right  and  left  arbitrary  multipliers 
in  the  rank  reduction  equation  and  write 


E  =  Z  -  Z  L  [(1/  Z')  Z  L]"1  (1/  Z')  Z 


(20.8.19) 


R  =  Z'  Z 


(20.8.20) 


we  have  from  Eqs.  (20.8.18),  (20.8.19),  and  (20.8.20) 
E  =  Z  -  (Z  L  t'*1)  (t1  L'  R) 


(20.8.21) 


* 
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We  now  consider  the  general  lower  rank  approximation  form  to  the 
data  matrix 

E  =  Z  -  Y  a'  (20.8.22) 

If  a  is  given  by  Eq.  (20.8.17),  then  from  Eq.  (20.8.21) 

Y  =  Z  L  t'"1  (20.8.23) 

and  from  Eqs.  (20.8.18),  (20.8.20),  and  (20.8.23) 

Y'  Y  =  I  (20.8.24) 

If  now  we  have  an  orthonormal  transformation 

b  .  a  H  (20.8.25) 

we  consider  W  such  that 

W  b'  =  Y  a'  (20.8.26) 

From  Eqs.  (20.8.25)  and  (20.8.26) 

W  =  YH'  (20.8.27) 

which  is  a  simple  structure  factor  score  matrix.  Since  H  is  orthonormal  by 
definition,  we  have  from  Eqs,  (20.8.24)  and  (20.8.27) 

W'  W  =  I  (20.8.28) 

If  the  simple  structure  transformation  is  not  orthonormal,  we  use  fa  instead 


* 
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of  H  and  show  that 


w  =  Y  h 


and  the  covariance  matrix  C  for  w  is 

w  — 


(20.8.29) 


V 


(b'  h) 


-l 


(20.8.30) 


or  if  we  let  r  be  the  correlation  of  the  simple  structure  reference  axes, 
then 


C  =  r 
w 


(20.8.31) 


For  computational  purposes  we  have  from  Eq.  (20.8.4) 
t'  =  a'  L 
Y  =  Z  (L  t'*1) 


(20.8.32) 

(20.8.33) 


Then 


W  =  Z(L(trlH')) 


for  the  orthonormal  transformation,  and 


w  =  Z  (L  (t'"1  h1)) 


(20.8.34)! 


(20.8.35) 


20,8.2  The  Multiple  Group  Factor  Score  Matrix 

Suppose  we  let  f  be  the  binary  grouping  matrix  for  the  multiple  group 
method  of  factor  analysis.  The  factor  loading  matrix  given  in  Chapter  6  is 


=  Eft' 


-1 


» 


a 


(20.8.36) 
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where 

t  t'  =  t*  R  f  •  (20.8.37) 

Then  by  the  methods  of  the  preceding  section  we  have  for  the  rank  reduction 
type  multiple  group  factor  score  matrix 

Y  =  Z  f  t'"1  (20.8.38) 

Since 

R  =  Z'  Z  (20.8.39) 

we  can  readily  see  from  Eqs.  (20.8.37),  (20.8.38),  and  (20.8.39)  that 
*.  Y'  Y  *  I  (20.8.40) 

We  also  show  that  Y  can  be  computed  directly  from  the  data  matrix*  We  let 

*  — 

U  =  Z  f  (20.8.41) 

tt'  •  U'  U  (20.8.42) 

From  Eqs.  (20.8.37),  (20.8.38),  (20.8.39),  (20.8.41),  and  (20.8.42) 

Y  =  U  t'"1  (20.8.43) 

We  may  now  use  Eqs.  (20.8.36),  (20. 8. 38),  and  (20.8.43)  to  calculate  a  from 
the  factor  score  matrix  Y,  thus 

a  =  Z'  Y  (20.8.44) 

« 

ft 


1?4 


The  same  procedure  could  have  been  used  to  calculate  the  centroid  factor 
score  matrix  directly  from  the  data  matrix,  having  given  the  matrix  L  of  sign 
vectors.  It  will  be  recalled  that  in  Chapter  5  the  successive  residual  matri¬ 
ces  were  required  from  which  to  calculate  the  successive  sign  vectors,  and 
hence  they  are  not  available  in  advance.  It  is  of  interest  to  note  that  a 
in  Eq.  (20.8.44)  is  precisely  a  matrix  of  the  correlations  of  the  factor 
scores  with  the  variables,  since  both  Z  and  Y  are  in  standard  measures.  For 
the  case  of  orthonormal  transformations  to  simple  structure  we  have 

W  =  Y  H  (20.8.45) 

and  for  oblique  transformations  we  have 

w  =  Y  h'"1  (20.8.46) 

For  the  computational  equations  we  have  Eqs.  (20.8.41),  (20.8.42),  and 

W  =  U  (t'"1  H)  (20.8.47) 

and 

w  =  U  (t'*1  h'*1)  (20.8.48) 

for  the  orthonormal  and  oblique  transformations,  respectively. 

20.8.5  The  Principal  Axis  Factor  Score  Matrix 

The  basic  structure  or  principal  axis  factor  score  matrix  is  well  known 

to  be 

Y  =  Z  U’1  (20.8.49) 


t 
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or,  if  the  basic  structure  of  Z  is 
Z  =  Pi  9' 


(20.8.50) 


then 


Y  =  P 


(20.8.51) 


For  the  orthonormal  and  oblique  transformation,  respectively,  we  have 


W  =  PH 


(20.8.52) 


and 


*5 


w  =  P  h'*1 


(20.8.55) 


For  the  covariance  matrices  of  Eqs.  (20.8. 51),  (20.8.52),  and  (20.8.55) 
we  have 


Cy  =  I  (20.8.54) 

cw  =  I  (20.8.55) 

Cw  =  (h'  h)'1  =  r"1  (20.8.56) 

20.8.4  The  Least  Square  Factor  Score  Matrix 

Given  the  arbitrary  factor  loading  matrix  a  and  the  residual  factor 
score  matrix 


* 


Z  -  Y  a' 


E 


(20.8.57) 


* 
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the  solution  for  Y  which  minimizes  tr  E'E  is  well  known  to  be 


Y  =  Z  a  (a'  a)*1 


(20.8.58) 


a  =  RLt' 


(20.8.59)- 


where  L  is  arbitrary  and 


R  =  Z7  Z 


(20.8.60) 


tt'  =  L7  R  L 


. (20.8.61) 


then  from.Eqs.  (20.8. 58)  through  (20.8. 6l) 

Cy  =  Y7  Y  *  t7  (L7  R2  L)"1  (L7  R5  L)  (L7  R2  L)'1  t  (20.8.62) 


R  =  Q  A2  Q7 


(20.8.63) 


L  =  Q  A 


(20.8.64) 


then  it  can  be  shown  that 


t  =  t7  =  A 


(20.8.65) 


and  therefore  also  that 


cY  =  I 


v 


(20.8.66) 
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There  appears,  however,  to  be  no  simple  expression  for  for  other  I  matri¬ 
ces,  such  as  the  centroid  sign  matrix,  or  the  binary  grouping  matrix  f. 

If  we  have  the  orthonormal  or  oblique  transformations  H  or  h,  respect¬ 
ively,  then 


W  =  Y  H 


(20.8.67) 


w  =  Y  h'" 


(20.8.68) 


For  the  case  of  Eq.  (20.8.64)  we  have 


CW  "  1 


(20.8.69) 


C  =  rJ 
w 


(20.8.70) 


Otherwise, 


CW  *  h'cyh 


(20.8.71) 


C  =  h"1  Cv  hri 
w  Y 


(20.8.72) 


as  can  be  seen  from  Eqs.  (20.8.62),  (20.8.67),  and  (20.8.68). 
20.8.9  The  Image  Analysis  Factor  Score  Matrix 


U  =  Z  (I  -  R  BjO  D 


(20.8.75) 
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be  the  image  data  matrix  scaled  with  P  according  to  one  of  the  procedures 
of  Chapter  15 . 

We  let 


M  -  (I  -  n  I§>)  D 


(20.8.710 


and  the  scaled  image  covariance  matrix  be 


C„  •  O'  0 


(20.8.75) 


Let  L  be  a  matrix  of  the  order  of  the  factor  matrix.  As  a  special  case,  it 
may  be  a  centroid  sign  matrix,  a  binary  grouping  matrix,  or  the  principal 
axis  factor  matrix  of  Cy.  In  all  cases,  we  know  by  the  previous  methods 
that  the  rank  reduction  type  factor  score  matrix  is  as  follows. 


u  =  U  L 


(20.8.76) 


tt'  =  u#  u 


(20.8.77) 


I  =  ut' 


(20.8.78) 


We  let 


U  =  PAQ' 


(20.8.79) 


L  =  Q  A 


(20.8.80) 


that  is,  L  is  the  principal  axis  factor  matrix  of  Cy.  Then  it  can  readily 
be  shown  from  Eqs.  (20. 8. 76)  through  (20.8.80)  that 


% 
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**  Y  =  P  (20.8.81) 


W  =  PE 

(20.8.82) 

w  =  P  h'*1 

(20.8.85) 

Cy  =  I 

(20.8.84) 

CW  =  1 

(20.8.85) 

Cw  =  (h'  h)*1  =  r1 

(20.8.86) 

the  general  case  of  L  we  have  computationally 

tt'  =  L'  D  (1  -  2  D^l  +  B|-l  R1  DjJd)  D  L 

(20.8.87) 

B  =  M  (L  t'"1) 

(20.8.88) 

Y  =  ZB 

(20.8.8 9) 

W  =  Z(B  H) 

(20.8.90) 

w  =  Z  (B  h'"1) 

(20.8.91) 

For  the  general  case  we  also  have  Eqs.  (20.8.84),  (20.8.85)/  and  (20.8.86). 


K 
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